Originally posted by Prashob Nadukandi
Hello All,
Went through the recordings of the AO modernization workshop session which covered activation groups to find a solution for a scenario currently being faced in a customer production environment.
Usually our programs are compiled with *caller activation group. And if it gets called from a never ending batch job, my understanding is that job will retain the instance of the program in its PAG. Changes done in the program logic at a later point doesn't become effective in that job and a recycle of the job is needed.
Will it be the same case even if I make that called program to now run in a *new activation group?
Recycling the jobs need downtime and mailers being sent out etc as those are critical jobs. So thinking if the new program changes can be made effective without recycling the job?
Thanks,
Prashob
Hi Prashob
When a program is changed (recompiled) while in use by a job, the program in use is copied to the QRPLOBJ library where it continues to be used until the job (or program) is terminated. The changed program will be used when the next call to that program causes it to be initialized.
From what you have said about the need to re-start the batch job I am assuming that the program being called does not end with a set-on of the LR indicator. If it does, then the next time it is called, the program would be re-initialized and the problem you are describing would not exist.
Your other suggested solution of creating a *NEW activation group for this program to run in would work just as well, as on exit from the program all resources would be recovered and the program would be closed down. Then on recall the initialization would use the new program.
Remember, using a new activation group might introduce additional complications, such as commitment control. Review the additional implications of the activation group route before choosing this option, although in my opinion it is the better option.
I would suggest a test of both these possible solutions first, to minimize the need to recycle the batch job unnecessarily.
Without knowing exactly how this batch job is operating, I apologize for not being able to be more specific.
Prashob Nadukandi replied:
Came up with a simple test programs with below logic
1. Write a simple batch program 1 which has a delay loop of 1 min and calls a program 2. The loop iterates for 10 times .Compile the program 1 with activation group *new
2. Write a simple program 2 which updates a data area with the current date, current time and a literal ‘ This is Version 1’.
Ex : 2016-10-13 12:12:12 This is Version 1
Compile the program 2 with activation group *caller.
3. Run the batch job which calls the program 1. Expected result is after every min the data are should get updated with current date, current time and a literal ‘ This is Version 1’.
Test Result : As expected
4. Modify the program 2 to change the literal to ‘This is Version 2’. Expected result is after every min the data are should still get updated with current date, current time and a literal ‘ This is Version 1’.
Test Result : As expected
5. Now recompile the program with activation group *new and check how the data area gets updated after every min.
Test Result : The data area is still getting populated with the literal present at the it was submitted even though the program 2 was recompiled with new literals while the batch job was still active.
6. Restart the batch job. Expected result is after every min the data area should get updated with current date, current time and the latest literal.
Test Result : The data area is still getting populated with the literal present at the it was submitted even though the program 2 was recompiled with new literals while the batch job was still active.
The activation group of program 2 in the call stack did change after every subsequent call.
The service debug of the batch job resulted in the break point set in program 2 being hit only once and didn’t occur in the subsequent calls.
Once the program 2 got recompiled it started using the object from QRPLOBJ library. The object remained same even though the activation group kept changing and also further recompiles were done when the batch job was active.
That explains why the break point didn't function in service debug after the called program was recompiled.
The called program does set the *inlr on.
Prashob Nadukandi replied:
TSTPGM1 :
H dftactgrp(*no) actgrp(*new)
H option(*srcstmt:*nodebugio)
H bnddir('QC2LE')
D Sleep Pr 10u 0 extproc('sleep')
D Interval 10u 0 value
D TstPgm2 Pr ExtPgm('TSTPGM2')
D I S 2 0
/Free
For i = 1 to 10;
TstPgm2();
sleep (60);
Endfor;
*Inlr = *On;
/End-Free
-----------------------------------------------------------------
TSTPGM2 :
H dftactgrp(*no) actgrp(*NEW)
H option(*srcstmt:*nodebugio)
H bnddir('QC2LE')
D TstDtaara Ds 100 Dtaara(TstDtaara)
D TstPgm2 Pr ExtPgm('TSTPGM2')
D Sleep Pr 10u 0 extproc('sleep')
D Interval 10u 0 value
D TstPgm2 PI
/Free
In *Lock TSTDTAARA;
TSTDTAARA = %Char(%date():*Iso) +' '+ %Char(%Time():*HMS) +
' This is version 8';
Out TSTDTAARA;
Sleep(10);
*Inlr = *On;
/End-Free
-----------------------------------------------------------------
SBMJOB CMD(CALL PGM(TSTPGM1)) JOB(PRASHOBTST)
Prashob Nadukandi replied:
Hi All,
Did further testing and this time introduced second level program call TSTPGM3.
Test result : even if tstpgm2 and tstpgm3 are compiled with *caller activation group always the latest version of TSTPGM3 got picked up!!!
TSTPGM1 :
H dftactgrp(*no) actgrp(*new)
H option(*srcstmt:*nodebugio)
H bnddir('QC2LE')
D Sleep Pr 10u 0 extproc('sleep')
D Interval 10u 0 value
D TstPgm2 Pr ExtPgm('TSTPGM2')
D I S 2 0
/Free
For i = 1 to 10;
TstPgm2();
sleep (20);
Endfor;
*Inlr = *On;
/End-Free
-----------------------------------------------------------------
TSTPGM2 :
H dftactgrp(*no) actgrp(*caller)
H option(*srcstmt:*nodebugio)
D TstDtaara Ds 100 Dtaara(TstDtaara)
D TstPgm2 Pr ExtPgm('TSTPGM2')
D TstPgm3 Pr ExtPgm('TSTPGM3')
D TstPgm2 PI
/Free
In *Lock TSTDTAARA;
%SubSt(TSTDTAARA:1:19) = %Char(%date():*Iso) +' '+ %Char(%Time():*HMS);
Out TSTDTAARA;
TstPgm3();
*Inlr = *On;
/End-Free
-----------------------------------------------------------------
TSTPGM3 :
H dftactgrp(*no) actgrp(*caller)
H option(*srcstmt:*nodebugio)
H bnddir('QC2LE')
D TstDtaara Ds 100 Dtaara(TstDtaara)
D TstPgm3 Pr ExtPgm('TSTPGM3')
D Sleep Pr 10u 0 extproc('sleep')
D Interval 10u 0 value
D Literal C Const('This Version 2')
D TstPgm3 PI
/Free
In *Lock TSTDTAARA;
%SubSt(TSTDTAARA:21) = Literal;
Out TSTDTAARA;
Sleep(10);
*Inlr = *On;
/End-Free
-----------------------------------------------------------------
SBMJOB CMD(CALL PGM(TSTPGM1)) JOB(PRASHOBTST)
Prashob
I have replicated all your testing (all day today) and agree with every one of your findings. This is a very strange situation, especially with the 3 program stack. I does not make sense.
I even went so far as to insert an *INZSR sub-routine in PGM 2 & 3 to make sure that the programs were initialized on each call, which they are when using *new as the activation group, as they should be, even when PGM2 is called from the QRPLOBJ schema every time and program 3 is not.
I am going to continue to work on this to find a solution as this is not correct!
Prashob Nadukandi replied:
Is there a way to post screenshots on this forum. That would give a more clear understanding
Activation Group from Never Ending Batch Job
This entry details the testing and results from the three-tier call from a never-ending batch program and specifically identifies the anomaly encountered around the first call.
The Coding was created to minimize the code set and to allow the tests to be run using the interactive debug facility available in RDi as the test tool.
TSTPGM1
*===============================================================
h dftactgrp(*no) actgrp('LRBATCH') usrprf(*owner) aut(*use)
*===============================================================
d TstPgm2 pr extpgm('TSTPGM2')
*===============================================================
d I s 5i 0
*===============================================================
/free
//=============================================================
for I = 1 to 10;
TstPgm2();
endfor;
//=============================================================
*inlr = *on;
//=============================================================
/end-free
*===============================================================
TSTPGM2
*===============================================================
h dftactgrp(*no) actgrp(*new) usrprf(*owner) aut(*use)
*===============================================================
d TstDtaara ds 100 dtaara(TSTDTAARA)
d Lit2 c 'PGM2A'
d TstPgm3 pr extpgm('TSTPGM3')
d Count s 5i 0
*===============================================================
/free
//=============================================================
In *LOCK TSTDTAARA;
TSTDTAARA = %Char(%timestamp():*ISO) + ' ' + Lit2;
Out TSTDTAARA;
//=============================================================
TstPgm3();
//=============================================================
return;
//=============================================================
begsr *inzsr;
Count = 1;
endsr;
//=============================================================
/end-free
*===============================================================
TSTPGM3
*===============================================================
h dftactgrp(*no) actgrp(*new) usrprf(*owner) aut(*use)
*===============================================================
d TstDtaara ds 100 dtaara(TSTDTAARA)
d Lit3 c 'PGM3A'
d Count s 5i 0
*===============================================================
/free
//=============================================================
In *LOCK TSTDTAARA;
TSTDTAARA = %trim(TSTDTAARA) + ' ' + Lit3;
Out TSTDTAARA;
//=============================================================
return;
//=============================================================
begsr *inzsr;
Count = 1;
endsr;
//=============================================================
/end-free
*===============================================================
A data area “TSTDTAARA” of *CHAR(100) needs to be created in the schema being used for testing.
An RDi debug entry point needs to be set for TSTPGM1 before being submitted to the job queue using
“SBMJOB CMD(CALL PGM(TSTPGM1)) JOB(TESTJOB)”.
Stepping into the programs 2 & 3 through a couple of cycles of TSTPGM1, it all looks normal and each entry into TSTPGM2 and 3 passes through the *INZSR sub-routine indicating that the previous exit from the *new activation group has in fact recovered the program resources on exit, as is expected.
One can see from the debug call stack below, that the correct programs are being opened from the correct schema.
tmp [IBM i: Debug Job]
Platform: IBM i Connection: 41-134-221-149.dsl.mweb.co.za:141
Thread:1 (stopped)
TSTPGM3 : TSTPGM3 : TSTPGM3
_QRNP_PEP_TSTPGM3 : TSTPGM3 : TSTPGM3
TSTPGM2 : TSTPGM2 : TSTPGM2
_QRNP_PEP_TSTPGM2 : TSTPGM2 : TSTPGM2
TSTPGM1 : TSTPGM1 : TSTPGM1
_QRNP_PEP_TSTPGM1 : TSTPGM1 : TSTPGM1
QCMD : QCMD : QCMD
Process: 049134/TOMMYA/TESTJOB Program: TSTPGM1
The next thing is to stop stepping the debug when in TSTPGM1, change the literal in both program 2 & 3 from “A” to “B” and recompile both TSTPGM2 and TSTPGM3.
Step into TSTPGM2 and notice that the program is entered at the *INZSR sub-routine indicating the program is re-initialized, however if one looks at the debug stack (below) it is clear that the program is opened from the QRPLOBJ schema with a generated program name, caused by the recompile while the job is active, which is acceptable EXCEPT the program was not active, otherwise it would not have been re-initialized.
tmp [IBM i: Debug Job]
Platform: IBM i Connection: 41-134-221-149.dsl.mweb.co.za:141
Thread:1 (stopped)
TSTPGM2 : Q878DE4C5A : TSTPGM2
_QRNP_PEP_TSTPGM2 : Q878DE4C5A: TSTPGM2
TSTPGM1 : TSTPGM1 : TSTPGM1
_QRNP_PEP_TSTPGM1 : TSTPGM1 : TSTPGM1
QCMD : QCMD : QCMD
Process: 049134/TOMMYA/TESTJOB Program: TSTPGM1
Continue to step into program 3 and notice once again the *INZSR is called.
Although Program 3 has been subjected to an identical change to Program 2 the correct (changed) program is being initialized as can be seen from the following call stack.
tmp [IBM i: Debug Job]
Platform: IBM i Connection: 41-134-221-149.dsl.mweb.co.za:141
Thread:1 (stopped)
TSTPGM3 : TSTPGM3 : TSTPGM3
_QRNP_PEP_TSTPGM3 : TSTPGM3 : TSTPGM3
TSTPGM2 : Q878DE4C5A: TSTPGM2
_QRNP_PEP_TSTPGM2 : Q878DE4C5A: TSTPGM2
TSTPGM1 : TSTPGM1 : TSTPGM1
_QRNP_PEP_TSTPGM1 : TSTPGM1 : TSTPGM1
QCMD : QCMD : QCMD
Process: 049134/TOMMYA/TESTJOB Program: TSTPGM1
Step program 3 to the “RETURN” line and the examine the TSTDTAARA and it can be clearly see that the change made to program 2 has NOT been applied whereas the change made to program 3 HAS been applied.
This is a totally unexpected result and needs further investigation to determine the cause of this anomaly. These test results will be forwarded to IBM for an opinion and hopefully a resolution.
If the test is repeated but a change is made only to program 3 and not to 2 then everything works as expected and the change from program 3 is correctly placed into the data area.
This technique of inserting a dummy program into the call stack under these conditions could be used as a possible, but less than satisfactory, work-around until a prober solution is found.
Prashob, you need to create the screen images as *.jpg files and then use the "Image" button in the editing bar at the top of the message box to insert the images.
A PMR has been registered with IBM, as this seems to be a software malfunction.
PMR number is: 27629,999,864
Joe Guetzlaff replied:
Here are 3 CL prog-samples, that will enable the behaviour you're looking for.
T$STK00
PGM
DCLPRCOPT DFTACTGRP(*NO) ACTGRP(*CALLER)
TAG01:
CALL PGM(T$STK01)
DLYJOB DLY(10)
GOTO CMDLBL(TAG01)
ENDPGM
T$STK01
PGM
DCLPRCOPT DFTACTGRP(*NO) ACTGRP(*NEW)
CALL PGM(T$STK02)
ENDPGM
T$STK02
PGM
DCLPRCOPT DFTACTGRP(*NO) ACTGRP(*CALLER)
SNDMSG MSG('Testmessage') TOUSR(JOE)
ENDPGM
When T$STK01 ends, which is the only PGM to establish a new ActGrp, the ActGrp ends and all resources are freed. Hence T$STK02 will be freshly invoked with the next call!
When looking at activation groups, you look at a kind of container. Using INLR or Return within a RPG program makes no difference in this context. INLR closes all files and cleans up internally declared resources, but is invisible to the caller, which would be responsible to free/dispose it's allocated resources, as such. Actually there is a FREE opcode in fixed-format RPG, which was supposed to "unload" a called program within the OPM environment.
Except for very special reasons, I cannot recommend using named activation groups at all. Simply too error-prone!
If you don't already do so, I recommend using ILE-CL (CLLE) for easy control of the call stack and for call stack entry programs. A RPG will mostly be compiled with ACTGRP(*CALLER), a CLLE (ILE-CL) can then be used to force this RPG to run within a virgin environment. A RPG program should never have to worry about it's technical environment.
Hope that helps! Happy programming!
Joe Guetzlaff replied:
For testing purposes:
I forgot to add, that while T$STK00 is running/active, you can change the message string in T$STK02, recompile and the new message will be send!
You can even invoke T$STK00 interactively - and cancel with sysreq-2 🙁
Hi Joe!
GOOD of you to visit and participate!!
Joe, I am amazed that you provided me with such an opportunity to tease you! 
This group is dedicated to RPG, not CL... 
So we would prefer RPG based solutions, although we do realise that some can be achieved using other languages... 
Thanks for providing me with a moment of mirth!!
Have a GREAT day!
Hi Joe
Thank you for providing me with the perfect opportunity to facilitate a discussion on how we will attempt to manage the group. 
It is ABSOLUTELY focused and RPG (and ILE) centric. That is why I teased you, as I do understand WHY you used CL.
We ABSOLUTELY do have SQL, CL, COBOL, C and many other tools in our toolbox or arsenal, but we will take a DECIDED perspective on a RPG (and data - hence DDL and DML) centric view. 
All contributions will be measured against that; is this a (for instance) DB2 <-> RPG <-> JAVA question and how to exploit the best from each, or is it a (for instance) JAVA centric question. If RPG (and ILE) is not fundamental to the discussion, we respectfully suggest it does not belong here - there are MANY forums that caters for that audience.
This site is DEDICATED to get MAXIMUM out of RPG in the ILE paradigm.
We WILL assist and "tolerate" LIMITED OPM discussions, but with ILE having being around for 22+ YEARS, people should not even consider OPM for new development.
We should perhaps ask Jim Buck to chime in on the "RPG overhead" issue (run-time), as he quite recently wrote an EXCEPTIONAL article on the subject. I will ask him to post it in the forums, as it was such a great piece of information...
Keep them coming!! 
I raised a PMR with IBM to get to the bottom of this issue and post below the response received from IBM as well as the follow-up questions and answers.
IBM’s Initial response (from Keith);
What you are seeing here is working as designed and I would not recommend this as a work around to recycling a never ending job when program changes are made.
Details:
PGM1 runs in named activation group(AG) LRBATCH
PGM2 and PGM3 run in AG *NEW
On return; from PGM2 and PGM3, these AG's are cleaned up by the operating system, as they no longer have an active program. Since LRBATCH never ends, it remains active. Since LRBATCH never ends, it still has a valid pointer from the previous call to PGM2. Thus if you recompile PGM2 after making a change, PGM1 will still point to the version in QRPLOBJ since it remembers the previous call.
If you make a change to PGM3 as well, PGM2 will call the new version of the program, since the previous AG group had been cleaned up and you are calling from a brand new AG. PGM2 has no memory of ever previously calling PGM3, so it picks up the new pointer.
If you wanted to pick up changes made in PGM2 and PGM3, you would need an intermediary PGM1A that simply calls PGM2.
This type of programming is bad for 2 reasons:
- *NEW is resource intensive because it needs to regenerate everything needed for the program on each call
- Since this is a never ending job, you would eventually reach the maximum number of AGs for a job, and the entire application would fail.
Confirmation Question to IBM;
LRBATCH “remembers” the call to PGM2 which is in QRPLOBJ due to the recompile but it was closed down on exit from the AG so it is the OLD program that is reinitialized when it is recalled which is why it executes the *INZSR and other initialization routines.
Is this correct?
Response from Eric;
Keith is out of the office until Tuesday, I am covering this PMR while he is out. I worked with him on this when it came in this morning, so I am familiar with it.
Yes, your understanding is correct.
Follow-Up Questions;
I would like to ask 2 more questions, after which I will be happy for you to close the PMR.
- What is the actual maximum number of *NEW AG’s for a job.
- Would the use of a ‘named’ activation group avoid 1. above, and would it change the behavior of the LRBATCH job if the batch job was triggered, via a *MSGQ message, to issue a RCLACTGRP for the ‘named’ AG when a program had been changed.
Answers from Keith;
What is the actual maximum number of *NEW AG’s for a job.
- A job can have up to 4,294,967,295 activation groups.
Would the use of a ‘named’ activation group avoid 1. above,
- If you were testing each time to see if the program had been changed, and only reclaimed the named AG if the program had changed, it would not 'avoid 1' altogether. The system would still need to restart the AG each time RCLACTGRP is issued, although it would be better than creating a new AG on every program call.
Would it change the behavior of the LRBATCH job if the batch job was triggered, via a *MSGQ message, to issue a RCLACTGRP for the ‘named’ AG when a program had been changed.
- You would need to test this, we have no way of knowing how your application would respond.
- Again, IBM recommends recycling jobs using a program to pick up changes that were made. This ensures no remnants to the old object remain in the job. If you elect to circumvent this, thorough application design and testing will be required.
Hope this gives some insight into the issue and helps to overcome the problems.
My personal solution, were I to be faced with this problem, would be to use the “Named Activation Group” option where the RCLACTGRP is triggered by a message arriving in a message queue.
This would still require the insertion of a dummy program (2) before the actual program (3).
Slaninaj replied:
Tommy,
I have been look for a ways to do this and everyone I ask has said it can not be done.
How would your dummy program look and where would put put the trigger ?
We have RPGLE CGI Web service that need to know when to reclaim an activation group because of a newer program.
The "Dummy" program would be called by the batch program whenever it was required and will start a named AG using, as an example, actgrp('AG1'). This would mean that if the AG was already active no initialize would occur. This "Dummy" program would then simple pass the call onto the correct program required.
The batch job would contain the trigger code and would check for a message in a special message queue before making the call to the "Dummy". Use the "Receive Program Message (QMHRCVPM)" API with a wait time of 0 (no wait) and a message action of *REMOVE.
If a message is found, then reclaim the activation group "RCLACTGRP ACTGRP(AG1)" before calling the "Dummy" program again.
When you have changed a program, send a message to the message queue. You could even include the name of the AG in the sent message, if many are involved. This would tell the batch job which AG needs to be reclaimed.
This should work in theory, but please test well before implementing.
Also PLEASE note that an exit from an activation group caused by the sending of an *ESCAPE message will automatically reclaim the AG in question.
Brian Rusch replied:
An alternative "dummy" program technique is to use a variable program name and change the program name in the variable so the pointer to the program gets re-resolved on each call. DUMMY can be a CL program that does nothing, but it has to exist for this technique to work. I believe this would eliminate the requirement to reclaim the activation group. So TSTPGM1 would look like this:
H dftactgrp(*no) actgrp(*new)
H option(*srcstmt:*nodebugio)
D Sleep Pr 10u 0 extproc('sleep')
D Interval 10u 0 value
D TstPgm2 Pr ExtPgm(pgmName)
D pgmName S 21a Inz
D I S 2 0
/Free
For i = 1 to 10;
pgmName = 'TESTPGM2';
TstPgm2();
pgmName = 'DUMMY';
TstPgm2();
sleep (60);
Endfor;
*Inlr = *On
/End-Free
