Resolve error when logging in to Customer View from Visual Dialogue in Portrait Dialogue

Product Feature: Login issues
Operating System: Not stated
Database: Not stated
Configuration: Not stated


In Portrait Dialogue (PD), when attempting to log in to Customer View from Visual Dialogue, there is a long delay, and then the error (see below) appears in the browser window.  

There are no Portrait Dialogue (PD) error log entries related to this event, and nothing useful in the Windows Event Logs.

The IIS logs have the following error codes when there are attempts to access Customer View.  
301 - Moved permanently.
302 - Object moved.
401 - Access denied.
500 - Internal server error. 

In the PD API Trace there are very high call times shown when logging in to Customer View.
There are high call times for several "TMHCustomerAPI.GetLookupDataset" calls to the API.
There are also consistent high call times for a specific lookup field "CustDomainID=1000 LookupSourceName=LookupFieldName".

Server Error in '/MHCustomerView' Application.

The operation has timed out
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 

Exception Details: System.Net.WebException: The operation has timed out

Source Error:
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace:
[WebException: The operation has timed out]
   System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request) +251
   System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(WebRequest request) +4
   System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters) +180
   MH.DataAccessLayer.CustomerWebService.CustomerAPIService.GetLookupDataset(String ServerSession, Int32 CustDomainID, String LookupSourceName) in d:\Build\60\Sources\DotNet\DataAccessLayer\Web References\CustomerWebService\Reference.cs:624
   MH.DataAccessLayer.ServiceProxies.CustomerServiceProxy.MH.DialogServerApi.ApiServiceContracts.ICustomerApiService.GetLookupDataset(String serverSessionKey, Int32 domainID, String lookupSourceName) in d:\Build\60\Sources\DotNet\DataAccessLayer\ServiceProxies\CustomerServiceProxy.cs:643
   MH.DialogServer.DialogServer.GetLookupDataSetFromDialogServer(String sessionKey, String methodName, Int32 domainID, String lookupSourceName) in d:\Build\60\Sources\DotNet\Old\WebForms\WebFramework\DialogServer\DialogServer.cs:2758
   MH.DialogServer.DialogServer.GetLookupDataSet(String sessionKey, Int32 domainID, String lookupSourceName, Boolean refreshCache, Int32 absoluteCacheExpiryDuration) in d:\Build\60\Sources\DotNet\Old\WebForms\WebFramework\DialogServer\DialogServer.cs:2717
   MHCustomerView.UserControls.SearchNormal.CreateComboLookup(DomainGroup Grp, GroupField field) in d:\Build\60\Sources\DotNet\Old\WebForms\CustomerView\UserControls\SearchNormal.ascx.cs:557
   MHCustomerView.UserControls.SearchNormal.CreatePage(Int32 pageNum, GroupFields groupFields, DomainGroup group) in d:\Build\60\Sources\DotNet\Old\WebForms\CustomerView\UserControls\SearchNormal.ascx.cs:312
   MHCustomerView.UserControls.SearchNormal.CreateTabs() in d:\Build\60\Sources\DotNet\Old\WebForms\CustomerView\UserControls\SearchNormal.ascx.cs:155
   MHCustomerView.UserControls.SearchNormal.Page_Load(Object sender, EventArgs e) in d:\Build\60\Sources\DotNet\Old\WebForms\CustomerView\UserControls\SearchNormal.ascx.cs:112
   System.Web.Util.CalliEventHandlerDelegateProxy.Callback(Object sender, EventArgs e) +51
   System.Web.UI.Control.OnLoad(EventArgs e) +92
   System.Web.UI.Control.LoadRecursive() +54
   System.Web.UI.Control.LoadRecursive() +145
   System.Web.UI.Control.LoadRecursive() +145
   System.Web.UI.Control.LoadRecursive() +145
   System.Web.UI.Control.LoadRecursive() +145
   System.Web.UI.Control.AddedControl(Control control, Int32 index) +260
   System.Web.UI.ControlCollection.Add(Control child) +86
   MH.Web.UI.WebControls.MHPanel.CreateChildControls() in d:\Build\60\Sources\DotNet\Old\WebForms\Common\WebControls\MHPanel.cs:294
   System.Web.UI.Control.EnsureChildControls() +83
   System.Web.UI.Control.PreRenderRecursiveInternal() +42
   System.Web.UI.Control.PreRenderRecursiveInternal() +155
   System.Web.UI.Control.PreRenderRecursiveInternal() +155
   System.Web.UI.Control.PreRenderRecursiveInternal() +155
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +974


When memory is saturated and the OS begins paging, performance drops dramatically.
When there is not enough CPU power, processes back up in a queue, waiting to be processed.

Where we have seen this issue has been on servers which were under-sized for the job.  
  • Using Dynamic RAM, starting with 1GB
  • 2GB or 4GB RAM
  • 1 or 2 cores
In combination with this, there were processes that used a lot of CPU and RAM
  • Complicated Domain query
  • Many data groups, some with full table scan
  • Many lookup fields in the Data Groups with resource hungry SQL
  • A lot of fields in the Domain, and most of them marked to show in the web apps (Customer View)


UPDATED: April 20, 2017
Two things that can, and should be done, are:
  • Revise the SQL used in the Domain and DataGroups queries to make them as efficient as possible.
  • Raise the resources of the server to a supported level.
    • 8GB RAM and 4 Cores should be considered a minimum configuration,
    • 16GB RAM and 8 Cores is a preferred, recommended configuration.
These suggestions resolve this problem in most cases.