Executing Not Thread-Safe Projects

Post any question you may have in regards to GoAnywhere Director and let our talented support staff and other users assist you.
2 posts Page 1 of 1

RElliott63

Posts: 14
Joined: Thu Jul 01, 2010 10:42 am

Post by RElliott63 » Wed Nov 30, 2011 11:50 am
Not Thread-safe projects can cause issues when documents received must be processed in order, but not allowed to run concurrently. The option below will help you get around the this error:
ERROR Project 'MyProject' is defined as not thread safe and an instance with job number '1321992906829' is already running. Please wait for this job to finish and then try again.
The job has already errored out and stopped. The current job completes and the next file doesn’t trigger the Project again to continue. How do you get the inbound file triggered by GA Services to process without manual intervention?

I use 2 Project Utilities to accomplish this:

*Get Project Status*
Code: Select all
<!-- ************************************************************************ -->
<!-- Project:      Get Project Status                                         -->
<!-- Date Written: 06.09.2011                                                 -->
<!-- Scripted by:  Rick Elliott                                               -->
<!--                                                                          -->
<!-- Description:  Check to see if Project is running before executing        -->
<!--                                                                          -->
<!-- Called by:    Non Thread-safe jobs                                       -->
<!--                                                                          -->
<!-- Vars Used:                                                               -->
<!--     pFolder - Folder Name containing Project (Example:  /Dev)            -->
<!--    pProject - Project Name to check                                      -->
<!--     pActive - Return Var - Yes/No                                        -->
<!--                                                                          -->
<!-- *Notes*                                                                  -->
<!--    Depends on ResourceID (GADerby) PreDefined and pointing to the        -->
<!--    GoAnywhere Derby Database (/goanywhere/userdata/database/goanywhere)  -->
<!--                                                                          -->
<!-- ************************************************************************ -->
<project name="Get Project Status" mainModule="Main" version="1.0" threadSafe="true" logLevel="debug">

  <!-- Parms Passed -->
	<variable name="pFolder"  value="xxx" description="Project Folder" />
	<variable name="pProject" value="xxx" description="Project Name" />
	<variable name="pActive"  value=" "   description="Return Value - (Yes) or (No)" />

  <module name="Main" dependsOn="Check Parms,Check Project">

    <print label="Status - Successful" version="1.0" executeOnlyIf="'${CheckJob}' eq 'Yes'">
      <![CDATA[
*==========================================================================*
* Project..
* 
*   Folder:   ${pFolder}
*   Project:  ${pProject}
*
*   Status:   ${pActive}        
*==========================================================================*
      ]]>
    </print>

    <print label="Status - Invalid Parms" version="1.0" executeOnlyIf="'${CheckJob}' eq 'No'">
      <![CDATA[
*==========================================================================*
*     
* Invalid Parameters Passed:
* 
*     Folder:   ${pFolder}
*     Project:  ${pProject}
*        
*==========================================================================*
      ]]>
    </print>

  </module>

  <module name="Check Parms">

    <setVariable label="Set CheckJob" version="1.0" name="CheckJob" value="Yes" />
    <setVariable label="Check Parms"  version="1.0" name="CheckJob" value="No" executeOnlyIf="( '${pFolder}' eq 'xxx' )  or ( '${pProject}' eq 'xxx' )"/>
    <setVariable label="Set pActive"  version="1.0" name="pActive"  value="No"  />

  </module>

  <module name="Check Project" executeOnlyIf="'${CheckJob}' eq 'Yes'" >

    <setVariable label="Set statusError" version="1.0" name="statusError" value="0" />
    <sql label="Check GADerby" resourceId="GADerby" version="1.0" onError="setVariable:statusError=1" >
      <query label="See if Active" outputVariable="OutPut" whenNoDataFound="error">
        <statement>
Select Count(*) 
  From APP.DPA_JOB 
 Where Proj_Folder = &apos;/${pFolder}&apos; 
   and Proj_Name   = &apos;${pProject}&apos;
   and Status      = &apos;A&apos; 
        </statement>
      </query>
    </sql>

    <setVariable label="Set pActive" name="pActive" value="Yes" version="1.0" executeOnlyIf="${OutPut[1]} gt 0"/>

  </module>

	<description>Get Project Status</description>

</project>
*Run Project*
Code: Select all
<!-- ************************************************************************ -->
<!-- Project:      Run Project                                                -->
<!-- Date Written: 06.14.2011                                                 -->
<!-- Scripted by:  Rick Elliott                                               -->
<!--                                                                          -->
<!-- Description:  Check to see if Project is running before executing        -->
<!--               Project Name Passed                                        -->
<!--                                                                          -->
<!-- Called by:    Non Thread-safe jobs                                       -->
<!--                                                                          -->
<!-- Vars Used:                                                               -->
<!--     pFolder - Folder Name containing Project (Example:  /Dev)            -->
<!--    pProject - Project Name to check                                      -->
<!--                                                                          -->
<!-- *Notes*                                                                  -->
<!--    Checks to see if Folder/Project is currently active.  If it is,       -->
<!--    performs a delay for 'x' seconds over 'y' tries (Both variables).     -->
<!--    Each try waits 'x' seconds before trying again.  When Project is not  -->
<!--    active, it will submit the Project passed as a parameter as a Batch   -->
<!--    job for execution.                                                    -->
<!--                                                                          -->
<!--    Default values are 30 tries with 30 second intervals                  -->
<!--                                                                          -->
<!-- ************************************************************************ -->
<project name="Run Project" mainModule="Main" version="1.0" threadSafe="true" logLevel="debug">

  <!-- Passed Parms -->
	<variable name="pFolder"  value="xxx" description="Project Folder" />
	<variable name="pProject" value="xxx" description="Project Name"   />

  <module name="Main" dependsOn="Check Parms,Check Status,Run Project">

    <print label="Job Status" version="1.0">
      <![CDATA[
*==========================================================================*
* Project Status
*     Folder:           ${pFolder}
*     Project:          ${pProject}

*     Checked?          ${CheckJob}     (No = Error, or Invalid Parm Passed)
*     No/Tries:         ${Tries}        (How many checks for active project)
*==========================================================================*
      ]]>
    </print>

  </module>

  <module name="Check Parms">

    <setVariable label="Set CheckJob"    version="1.0" name="CheckJob" value="Yes" />
    <setVariable label="Check for Parms" version="1.0" name="CheckJob" value="No" executeOnlyIf="( '${pFolder}' eq 'xxx' ) or ( '${pProject}' eq 'xxx' )"/>

    <setVariable label="Set projDesc" name="projDesc" value="*Unidentifed*" version="1.0"/>

  </module>

  <module name="Check Status" executeOnlyIf="'${CheckJob}' eq 'Yes'" >

		<setVariable label="Set Active" version="1.0" name="Active" value="No" />
		<setVariable label="Set Tries"  version="1.0" name="Tries"  value="1" />
		
    <forLoop label="Check Project Status" beginIndex="1" endIndex="${MaxTries}" step="1" currentIterationVariable="Tries" >

      <callProject label="Get Project Status" version="1.0" project="/Utilities/Get Project Status" inheritUserVariables="true" returnUserVariables="true" >
  			<variable name="pFolder"  value="${pFolder}"  />
  			<variable name="pProject" value="${pProject}" />
  			<variable name="pActive"  value=" " />
  		</callProject>

      <setVariable label="Set Active" version="1.0" name="Active" value="${pActive}" />

      <exitLoop condition="'${Active}' eq 'No'" />

      <delay label="Try Delay" version="1.0" time="${Delay}" timeUnit="seconds" />
      
    </forLoop>
    
  </module>

  <module name="Run Project" executeOnlyIf="'${Active}' eq 'No'">

    <callProject label="Submit Job" project="/${pFolder}/${pProject}" runInSameJob="false" mode="batch" version="1.0" />

  </module>

	<variable name="MaxTries" value="30" description="How many times to check for job being active" />
	<variable name="Delay"    value="30" description="Delay in Seconds between checking for active job" />

	<description>Run Project checking for Thread-Safe</description>

</project>
Once these Utilities are in place (I usually put them into a /Utilities Folder within Director Projects) I can then trigger a call to the 'Run Project' project and pass the Folder & Project Parms from within the GoAnywhere Services - Action Tab. The Project to be triggered will be the '/Utility/Run Project' with 2 Parameters added -- Parm 1 : pFolder = 'MyFolder' -- Parm 2 : pProject = 'MyProject'

Effectively, this will call the 'Run Project' project that will validate whether there is already an instance of MyFolder/MyProject currently running. If there is, it will delay 'x' seconds and check again for 'y' tries (as identified above). This will allow the current active project to complete then execute the subsequent projects for processing instead of ending the job with the Not Thread Safe error message.

There are several tweaks you can make to this project to enhance it's abilities and provide conditional functionality. I say ... Go forth and Tweak!!

-Rick

RElliott63

Posts: 14
Joined: Thu Jul 01, 2010 10:42 am

Post by RElliott63 » Mon Mar 26, 2012 8:55 am
Minor Update to this Process:

I modified the ForLoop as follows:
Code: Select all
		<forLoop label="Check Project Status" beginIndex="1" endIndex="${MaxTries}" step="1" currentIterationVariable="Tries">

  		<setVariable label="Reset Delay Value" name="Delay" value="${ RandomNumber( 1, 12 ) }" version="1.0" />
  		<delay label="Delay Start" time="${Delay}" timeUnit="seconds" version="1.0" />

			<callProject label="Get Project Status" project="/Utilities/Get Project Status" inheritUserVariables="true" returnUserVariables="true" version="1.0">
				<variable name="pFolder"  value="${pFolder}"  />
				<variable name="pProject" value="${pProject}" />
				<variable name="pActive"  value=" " />
			</callProject>

			<setVariable label="Set Active" name="Active" value="${pActive}" version="1.0" />

			<exitLoop condition="${ Active  eq &apos;No&apos; }" />
    
			<delay label="Delay Start" time="20" timeUnit="seconds" version="1.0" />

		</forLoop>
This added a "random" number between 1 and 12 to allow for alternate start times if 2 or more documents are received at the same time. This should keep them from kicking off at the same time.

-R
2 posts Page 1 of 1