4

I would like some help identifying why this particular code, in rare circumstances, produces a race condition. I've found a fix, which I'll outline as well, but I really want to understand it.

We have a CMS based system comprised of many modules loosely based off a fusebox model. Everything runs through a single index.cfm.

In our Index.cfm we are creating a couple instances of Components, some are bsaed on the APPLICATION.PortalApp instance created in Application.cfc. I'm not including that code because it's not entirely relevant:

<cfset REQUEST.ActionHandler = CreateObject("Component", "Components.ActionHandler").init(APPLICATION.PortalApp.Config) />
<cfset VARIABLES.Modules = CreateObject("Component", "Components.Modules").init(APPLICATION.PortalApp.Config, REQUEST.ActionHandler.GetModuleList(), REQUEST.ActionHandler.GetSuppressOutput(), REQUEST.ActionHandler.GetRoleList(), REQUEST.ActionHandler.GetAccessList(), REQUEST.ActionHandler.GetMasterRoleList()) />

After we instantiate these objects, we get the content for the modules on the page (based on their 'pane': top, left, middle, right) by calling a PageManager component that's instantiated as part of the application, Application.PortalApp.

<cfsavecontent variable="Variables.Portal_Content.Top"><cfset APPLICATION.PortalApp.PageManager.DisplayContent( SESSION, REQUEST.ActionHandler, VARIABLES.Modules, 0  ) /></cfsavecontent>
<cfsavecontent variable="Variables.Portal_Content.Left"><cfset APPLICATION.PortalApp.PageManager.DisplayContent( SESSION, REQUEST.ActionHandler, VARIABLES.Modules, 1  ) /></cfsavecontent>
<cfsavecontent variable="Variables.Portal_Content.Middle"><cfset APPLICATION.PortalApp.PageManager.DisplayContent( SESSION, REQUEST.ActionHandler, VARIABLES.Modules, 2  ) /></cfsavecontent>
<cfsavecontent variable="Variables.Portal_Content.Right"><cfset APPLICATION.PortalApp.PageManager.DisplayContent( SESSION, REQUEST.ActionHandler, VARIABLES.Modules, 3  ) /></cfsavecontent>

PageManager.DisplayContent basically loops over the modules and wraps them in a wrapper. However, at some points, there is a race condition and the function craters and displays no module at all. It seems to be based on VARIABLES.Modules becoming corrupt but that's in the VARIABLES scope which is not shared.

To fix it, we changed the code to the following:

<!--- If we do not use VARIABLES scope and create a ContentManager, race condition can cause empty modules --->
<cfset VARIABLES.CurrPageMgr = CreateObject("Component", "Components.ContentManager").init() />

<cfsavecontent variable="Variables.Portal_Content.Top"><cfset VARIABLES.CurrPageMgr.DisplayContent( SESSION, REQUEST.ActionHandler, VARIABLES.Modules, 0  ) /></cfsavecontent>
<cfsavecontent variable="Variables.Portal_Content.Left"><cfset VARIABLES.CurrPageMgr.DisplayContent( SESSION, REQUEST.ActionHandler, VARIABLES.Modules, 1  ) /></cfsavecontent>
<cfsavecontent variable="Variables.Portal_Content.Middle"><cfset VARIABLES.CurrPageMgr.DisplayContent( SESSION, REQUEST.ActionHandler, VARIABLES.Modules, 2  ) /></cfsavecontent>
<cfsavecontent variable="Variables.Portal_Content.Right"><cfset VARIABLES.CurrPageMgr.DisplayContent( SESSION, REQUEST.ActionHandler, VARIABLES.Modules, 3  ) /></cfsavecontent>

The DisplayContent of ContentManager is the exact same function text as the PageManager.DisplayContent with the exception on ContentManager existing only in VARIABLES scope and PageManager existing as part of the APPLICATION.

This was very hard to reproduce after getting reports of it. I essentially started a jmeter session hammering the Development server as hard as possible with a breakpoint set based on a condition I knew would fire with VARIABLES.Module got corrupt. That was the only way to reproduce it.

I'm also not 100% sure this fix works, but so far jmeter has not fired the condition with it in place.

Edit: Per Request, DisplayContent function:

<cffunction name="displayContent" access="public" output="true">
    <cfargument name="SessionData" required="yes" type="Struct" />
    <cfargument name="ActionHandler" required="yes" type="ActionHandler" />
    <cfargument name="Modules" required="yes" type="Modules" />
    <cfargument name="Pane" required="yes" type="numeric" />
    <cfswitch expression="#Arguments.Pane#">
        <cfcase value="1"><cfset Variables.blnPane = ARGUMENTS.Modules.getLeft()></cfcase>
        <cfcase value="2"><cfset Variables.blnPane = ARGUMENTS.Modules.getCenter()></cfcase>
        <cfcase value="3"><cfset Variables.blnPane = ARGUMENTS.Modules.getRight()></cfcase>
        <cfdefaultcase><cfset Variables.blnPane = ARGUMENTS.Modules.getTop()></cfdefaultcase>
    </cfswitch>
    <cfif VARIABLES.blnPane>
        <cfset VARIABLES.qryPaneModules = ARGUMENTS.Modules.GetModulesInPane(Arguments.Pane)>
        <cfset VARIABLES.aryModulesInPane = ArrayNew(1)>
        <cfloop query="VARIABLES.qryPaneModules">
            <cfset VARIABLES.blnResult = ArrayAppend(aryModulesInPane,VARIABLES.qryPaneModules.MOD_SYS_NR)>
        </cfloop>
        <cfset VARIABLES.Template = "../CustomTags/Portalv#ARGUMENTS.SessionData.intPortalVersion#/DisplayModuleAlternate.cfm">
        <cfif Arguments.ActionHandler.GetDocumentType() EQ 3>
            <cfset VARIABLES.Template = "../CustomTags/Portalv#ARGUMENTS.SessionData.intPortalVersion#/DisplayXMLModule.cfm">
        </cfif>
        <cfif VARIABLES.qryPaneModules.recordcount GT 0>
            <cfloop index="VARIABLES.modLoop" from="1" to="#ArrayLen(VARIABLES.aryModulesInPane)#">
                <cfparam name="VARIABLES.aryModulesInPane[VARIABLES.modLoop]" default="0">
                <cfset VARIABLES.objModuleInfo = ARGUMENTS.Modules.GetModuleInfo( VARIABLES.aryModulesInPane[VARIABLES.modLoop], ARGUMENTS.Modules.GetModules() ) />
                <cfif NOT IsNumeric(VARIABLES.objModuleInfo.intModuleID)>
                    <cfset VARIABLES.objModuleInfo.intModuleID = 0 >
                </cfif>
                <cfmodule template="#VARIABLES.Template#"
                    ModuleID="#VARIABLES.objModuleInfo.intModuleID#"
                    ModuleName="#VARIABLES.objModuleInfo.strModuleName#"
                    SecurityLevel="#VARIABLES.objModuleInfo.intRoleTypeID#"
                    ModuleDSN="#VARIABLES.objModuleInfo.strModDBDSN#"
                    ModuleUserName="#VARIABLES.objModuleInfo.strModDBUserID#"
                    ModulePassword="#VARIABLES.objModuleInfo.strModDBPassword#"
                    AlternateFunctionID="#VARIABLES.objModuleInfo.intAlternateFunctionID#"
                    AlternateFunctionName="#VARIABLES.objModuleInfo.strAlternateFunctionName#"
                    InstructionFileID="#VARIABLES.objModuleInfo.intManualID#"
                    ModuleOps="#VARIABLES.objModuleInfo.blnModuleOps#"
                    ModuleSource="#VARIABLES.objModuleInfo.strModuleSource#"
                    ItemID="#VARIABLES.objModuleInfo.intModItemID#"
                    AutoLoginID="#VARIABLES.objModuleInfo.intAutoLoginCategoryID#"
                    IPS_objPortalSessionData="#ARGUMENTS.SessionData#"
                    ModuleList="#ARGUMENTS.ActionHandler.GetModuleList_SingleRecord(VARIABLES.objModuleInfo.intModuleID)#"
                    ModulesComponent="#ARGUMENTS..Modules#"
                    ControlHeaderIR = "#VARIABLES.objModuleInfo.blnControlHeaderIR#"
                    BorderIR = "#VARIABLES.objModuleInfo.blnBorderIR#"
                    IPS_strPortalRoot = "#VARIABLES.IPS_strPortalRoot#"
                    IPS_strPortalURL = "#VARIABLES.IPS_strPortalURL#"
                    Wrapper = "#Arguments.ActionHandler.getWrapper()#"
                    Definition = "#VARIABLES.objModuleInfo.strModuleDef#"
                    Category = "#VARIABLES.objModuleInfo.strModuleCat#"
                    aryModulesInPane = "#VARIABLES.aryModulesInPane#"
                    blnLockIR = "#VARIABLES.objModuleInfo.blnLockIR#"
                    strMessageTE = "#VARIABLES.objModuleInfo.strMessageTE#" >

            </cfloop>
        </cfif>
    </cfif>
</cffunction>
Brad
  • 1,684
  • 4
  • 20
  • 36
  • 1
    It seems likely that there is something (unscoped variable or missing lock etc) in `DisplayContent` that is causing the issue, especially as when you create it per request it seems to fix it. Would need to see the `DisplayContent` code to investigate further. – John Whish Apr 29 '16 at 16:25
  • @JohnWhish Added the function... – Brad Apr 29 '16 at 17:56
  • 1
    If that component is stored in the application scope, then storing something in its `variables` scope is essentially the same as storing it in the application scope. – Leigh Apr 29 '16 at 18:21
  • Leigh, I think that might be it... This line would be the issue: t When the component is instantiated in the VARIABLES scope there is no issue with that line, but running the DisplayContent that's a part of PageManager in the Application scope would have an issue. – Brad Apr 29 '16 at 18:23
  • 1
    I have not reviewed the whole thing, but I would think *anything* placed in the `variables` scope of `APPLICATION.PortalApp.PageManager` (Variables.blnPane, etcetera) is essentially in the application scope too. Seems like PageManager is a stateful component being used as if it were state-less, which would explain the race conditions, and why they go away when the component is *not* stored in the shared application scope. – Leigh Apr 29 '16 at 18:40
  • I changed the VARIABLES. to LOCAL. and I'm still getting the same issue... Shouldn't LOCAL scope variables within a function be thread-safe? – Brad Apr 29 '16 at 19:23
  • Disregard my previous comment, LOCAL scope does indeed seem to be working. There was an erroneous breakpoint set that fired when I first tested making me believe it was not fixed. – Brad Apr 29 '16 at 19:37
  • Be sure to [check any variables referenced by cfmodule](http://stackoverflow.com/questions/28631755/var-scoping-and-module-calls-in-a-cfc) as well. – Leigh Apr 29 '16 at 19:51
  • @Leigh - That would be a custom tag call, don't they always run within their own scope without fear of race conditions? – Brad May 02 '16 at 11:22
  • (Edit) @Brad - I would not say they are intrinsically safe, per se. All depends on what the CT is doing and "how". For example, CT's can modify the caller scope. In this case that would be PageManager.variables -> ie application scope, which would be prone to race conditions. – Leigh May 02 '16 at 13:34
  • This is not modifying the CALLER scope and jmeter testing is not triggering any race conditions with module not loading so I believe we are safe. Thanks for the heads up. – Brad May 02 '16 at 13:55

1 Answers1

1

Use of the VARIABLES scope inside the DisplayContent function of the PageManager component (which was instantiated as part of the Application, thus shared), was the issue. The VARIABLES scope would be shared in this case leading to race conditions.

Outside of duplicating that function in another component and instantiating it in the VARIABLES scope of the index.cfm, you could also switch from the VARIABLES scope in the DisplayContent function to the LOCAL scope (assuming you didn't need access to these variables outside the function). Both ways kept the race condition from re-appearing when stress testing with jmeter.

Brad
  • 1,684
  • 4
  • 20
  • 36
  • 1
    Yep. Writing to the `variables` scope makes the component state*ful*, and thus subject to race conditions when stored in a shared scope like application, because the `variables` scope is now accessible to multiple threads. Typically, only stateless components should be stored in shared scopes. For state*ful* components, usually you should create a separate, request level, instance so the variables are not shared/accessible to other requests. – Leigh May 02 '16 at 14:28