Monday, November 9, 2009

Make It Easy To Check User's Accessibility To PS Queries

In PeopleSoft, checking whether a user has access to a PS Query is sometime a very frustrating job, particularly when the query has references to many base records which are added into multiple access groups in different query trees, and these query trees and access groups are granted to different permission lists which are owned by different roles.

Unfortunately, I am the guy who have always been asked 'How come I am not able to view/edit query ... blah blah...' and I have spent too much time and efforts on this. Yesterday I eventually decided something must be done to pull me out of this repetitive and monotonous job, so I wrote the PL/SQL script chk_query_access.sql.

This script asks for 3 parameters:

1) PS Query Name: wildcards (%, _) are accepted. Escape character '\' is allowed too. For example, you can type in full query name N_Q006_SR_LOA, or you can also type in a query name pattern N\_Q00_\_% to check accessibility for queries N_Q001 to N_Q009.

2) User ID: is case-sensitive

3) Verbose Level: ranges from 0 - 3
  • Level 0: only shows query grant status. This is the default level.

SQL> @d:\SQL\chk_query_access.sql
SQL> SET ECHO OFF
Query Name (wilecard accepted): N_Q006_SR_LOA
User ID: PSTEST
Verbose Level:
0 - Show query grant status (default)
1 - Show query/record grant status
2 - Show query/record grant status and grant paths
3 - Show query/record grant status and all paths
Your Choice(0,1,2,3):
old 5: v_qryname_pattern PSQRYDEFN.QRYNAME%TYPE := trim('&prompt_qryname');
new 5: v_qryname_pattern PSQRYDEFN.QRYNAME%TYPE := trim('N_Q006_SR_LOA');
old 6: v_oprid PSOPRDEFN.OPRID%TYPE := trim('&prompt_oprid');
new 6: v_oprid PSOPRDEFN.OPRID%TYPE := trim('PSTEST');
old 7: v_verbose_lvl INTEGER := &prompt_verbose;
new 7: v_verbose_lvl INTEGER := 0;
===========================================================
=== Checking User(PSTEST)'s access to query 'N_Q006_SR_LOA'
===
===
=== Query 'N_Q006_SR_LOA' granted to 'PSTEST'
===

PL/SQL procedure successfully completed.
  • Level 1: shows query grant status and record grant status. This is useful when you want to know what record is not granted if the query is not accessible.
...
...
...
===========================================================
=== Checking User(PSTEST)'s access to query 'N_Q006_SR_LOA'
===
-----------------------------------------------------------
>>> Record 1: N_STNT_PERS_VW
>>> Record granted
-----------------------------------------------------------
>>> Record 2: N_STNT_SUMAC_VW
>>> Record not granted
-----------------------------------------------------------
>>> Record 3: N_LOA
>>> Record granted
===
=== Query 'N_Q006_SR_LOA' not granted to 'PSTEST'
===

...

  • Level 2: show query grant status, record grant status, and grant path. At this level the script also shows the whole grant path, eg Record - Query Tree/Access Group - Permission List - Role - User Profile.

...
...
...
===========================================================
=== Checking User(PSTEST)'s access to query 'N_Q006_SR_LOA'
===
-----------------------------------------------------------
>>> Record 1: N_STNT_PERS_VW
[Y] N_QUERY_TREE_RPT.N_RPTQAG_MOD_RANK -> N_R030_SR_MODULE_RANKING -> N_EXAM_QRY_EU -> PSTEST
[Y] N_QUERY_TREE_RPT.N_RPTQAG_BOE_ATTACH -> N_R031_SR_BOE_ATTACHMENTS -> N_EXAM_QRY_EU -> PSTEST
>>> Record granted
-----------------------------------------------------------
>>> Record 2: N_STNT_SUMAC_VW
>>> Record not granted
-----------------------------------------------------------
>>> Record 3: N_LOA
[Y] N_QUERY_TREE_DEN.N_DNQAG_LOA -> N_PROG_PLAN_ADMIN_QRY_EU -> N_PROG_PLAN_ADMIN_QRY_EU -> PSTEST
[Y] N_QUERY_TREE_DEN.N_DNQAG_LOA -> N_PROG_PLAN_ADMIN_QRY_IT -> N_PROG_PLAN_ADMIN_QRY_IT -> PSTEST
[Y] N_QUERY_TREE_RPT.N_RPTQAG_BOE_ATTACH -> N_R031_SR_BOE_ATTACHMENTS -> N_EXAM_QRY_EU -> PSTEST
>>> Record granted
===
=== Query 'N_Q006_SR_LOA' not granted to 'PSTEST'
===
...
  • Level 3: the most detailed verbose, especially useful when a query is not accessible and you need to find out at what position the granting is not done.

...
...
...
===========================================================
=== Checking User(PSTEST)'s access to query 'N_Q006_SR_LOA'
===
-----------------------------------------------------------
>>> Record 1: N_STNT_PERS_VW
[Y] N_QUERY_TREE_RPT.N_RPTQAG_MOD_RANK -> N_R030_SR_MODULE_RANKING -> N_EXAM_QRY_EU -> PSTEST
[Y] N_QUERY_TREE_RPT.N_RPTQAG_BOE_ATTACH -> N_R031_SR_BOE_ATTACHMENTS -> N_EXAM_QRY_EU -> PSTEST
>>> Record granted
-----------------------------------------------------------
>>> Record 2: N_STNT_SUMAC_VW
[N] N_QUERY_TREE_RPT.N_RPTQAG_MOD_RANK -> N_R030_SR_MODULE_RANKING -> N_EXAM_QRY_EU
[N] N_QUERY_TREE_RPT.N_ROGQAG_ENRL_STATS2 -> N_R042_SR_ENROL_STATS2
>>> Record granted
-----------------------------------------------------------
>>> Record 3: N_LOA
[Y] N_QUERY_TREE_DEN.N_DNQAG_LOA -> N_PROG_PLAN_ADMIN_QRY_EU -> N_PROG_PLAN_ADMIN_QRY_EU -> PSTEST
[Y] N_QUERY_TREE_DEN.N_DNQAG_LOA -> N_PROG_PLAN_ADMIN_QRY_IT -> N_PROG_PLAN_ADMIN_QRY_IT -> PSTEST
[Y] N_QUERY_TREE_RPT.N_RPTQAG_BOE_ATTACH -> N_R031_SR_BOE_ATTACHMENTS -> N_EXAM_QRY_EU -> PSTEST
>>> Record granted
===
=== Query 'N_Q006_SR_LOA' granted to 'PSTEST'
===


Updates:
  • 18-Nov-2009: Added check for defnition security.
  • 24-Nov-2009: Added support for command line arguments. Added access group cascading check.

Wednesday, November 4, 2009

Script Analyzing TraceSQL File And Extracting SQL Statements

TraceSQL is a great tool for Peoplesoft development debugging and application troubleshooting. But TraceSQL file only logs SQL statements and SQL variable values separately and so is less readable and hard to re-run.

This script is developed to analyze TraceSQL files, filter out unnecessary information, extract SQL statements and replace all SQL variables with the actual values.

For example, for the following contents in a tracesql file:


PSAPPSRV.12271 (951) 1-190 20.22.39 0.008245 Cur#1.12271.CS90SUP RC=0 Dur=0.000238 COM Stmt=SELECT OBJNAME, FLAG, PTCUSTOMFORMAT FROM PSUSEROBJTYPE WHERE MENUNAME = :1 AND PNLGRPNAME = :2 AND PNLNAME = :3 AND OPRID = :4 AND FIELDTYPE = :5
PSAPPSRV.12271 (951) 1-191 20.22.39 0.000013 Cur#1.12271.CS90SUP RC=0 Dur=0.000001 Bind-1 type=2 length=26 value=CALCULATE_TUITION_AND_FEES
PSAPPSRV.12271 (951) 1-192 20.22.39 0.000008 Cur#1.12271.CS90SUP RC=0 Dur=0.000000 Bind-2 type=2 length=14 value=ADJ_TERM_PANEL
PSAPPSRV.12271 (951) 1-193 20.22.39 0.000008 Cur#1.12271.CS90SUP RC=0 Dur=0.000001 Bind-3 type=2 length=1 value=
PSAPPSRV.12271 (951) 1-194 20.22.39 0.000006 Cur#1.12271.CS90SUP RC=0 Dur=0.000000 Bind-4 type=2 length=2 value=PS
PSAPPSRV.12271 (951) 1-195 20.22.39 0.000010 Cur#1.12271.CS90SUP RC=0 Dur=0.000000 Bind-5 type=18 length=2 value=-1

the script comes out with below SQL:


SELECT OBJNAME, FLAG, PTCUSTOMFORMAT FROM PSUSEROBJTYPE WHERE MENUNAME = 'CALCULATE_TUITION_AND_FEES' AND PNLGRPNAME = 'ADJ_TERM_PANEL' AND PNLNAME = ' ' AND OPRID = 'PS' AND FIELDTYPE = -1;


Script usage: xsql /path/to/tracesql

Update:
- 03-Mar-2010: Bug fix: encapsulated date/time values with quotes. Fixed the issue that the last SQL statement is not outputed.

Friday, October 2, 2009

Enhancements To The Scripts Listing Processes and Memory Usage of PeopleSoft Application Server and Process Scheduler

In the post released in February I introduced a script that is able to list all processes of a Peoplesoft app server/process scheduler as well as each process' memory usage information.

I have enhanced this script with the following new features:

- For Peoplesoft App Server & Process Scheduler:

= When -m is specified, the script will print CPU usage percentage aside from memory usage.


$ ~/bin/pl -cmh DOMAIN
PID PROCESS VSIZE(m) RSS(m) CPU%
--- ------- -------- ------ ----
7744 PSBRKHND 104.6 48.3 0.0
27887 PSAPPSRV 585.0 401.7 0.8
7799 PSPUBHND 100.7 24.6 0.0
7685 BBL 11.6 3.9 0.0
7704 PSSAMSRV 98.2 19.6 0.0
7796 PSBRKDSP 108.6 51.8 0.0
7802 PSPUBDSP 332.7 61.8 0.1
9122 JREPSVR 9.5 1.0 0.0
9055 JSL(9050) 11.0 1.9 0.0
8030 PSSUBHND 100.6 19.7 0.0
8350 PSSUBDSP 108.6 49.1 0.0
20335 PSAPPSRV 986.7 803.7 0.7
20562 PSAPPSRV 962.3 712.4 1.0
15765 PSAPPSRV 1006.4 805.3 0.5
13737 PSAPPSRV 188.0 157.3 0.8
6362 PSMONITORSRV 103.3 75.6 0.0
21579 PSWATCHSRV 14.6 8.2 0.0
9114 JSH(9053) 52.6 21.1 0.0
9089 JSH(9051) 44.6 16.8 0.1
9107 JSH(9052) 36.6 21.0 0.0



= When -s is specified, the script will print statistical information on the bottom.


$ ~/bin/pl -cmhs DOMAIN
PID PROCESS VSIZE(m) RSS(m) CPU%
--- ------- -------- ------ ----
7744 PSBRKHND 104.6 48.3 0.0
27887 PSAPPSRV 585.0 401.7 0.1
7799 PSPUBHND 100.7 24.6 0.0
7685 BBL 11.6 3.9 0.0
7704 PSSAMSRV 98.2 19.6 0.0
7796 PSBRKDSP 108.6 51.8 0.0
7802 PSPUBDSP 332.7 61.8 0.1
9122 JREPSVR 9.5 1.0 0.0
9055 JSL(9050) 11.0 1.9 0.0
8030 PSSUBHND 100.6 19.7 0.0
8350 PSSUBDSP 108.6 49.1 0.0
20335 PSAPPSRV 986.7 803.7 0.2
20562 PSAPPSRV 962.3 712.4 0.1
15765 PSAPPSRV 1006.4 805.3 0.3
13737 PSAPPSRV 188.0 157.3 0.1
6362 PSMONITORSRV 103.3 75.6 0.0
21579 PSWATCHSRV 14.6 8.2 0.0
9114 JSH(9053) 52.6 21.1 0.0
9089 JSH(9051) 44.6 16.8 0.1
9107 JSH(9052) 36.6 21.0 0.0
--- ------- -------- ------ ----
17 SERVER 4832.3 3245.7 0.9
5 PSAPPSRV 3728.4 2880.3 0.8
3 CLIENT 133.8 59.0 0.1


= For app server, ports opened by JSL/JSH/WSL/WSH will be printed.

See above example.


= When -r is specified, the script will print the summary memory and CPU usage of processes of all app servers/process schedulers running on the server. This is useful when you need to monitor server's performance. Actually Oracle recommends that the total resident memory for the entire PS Processes not exceed 70 percent of the total real memory available on the server.


$ ~/bin/pl -r
CATEGORY COUNT VSIZE(m) RSS(m) RSS% CPU%
-------- ----- --------- ------ ---- ----
All 98 37598.7 19232.7 29.3 1.7
PSAPPSRV 28 26224.3 15047.6 22.9 1.1
PSAESRV 0 0.0 0.0 0.0 0.0


- For Peoplesoft Web Server: the script now can print some configuration information and running information (memory and CPU usage) of a web domain.


$ ~/bin/pl -w webdomain
DOMAIN: webdomain
TYPE: Single Server
WEBSITES: server1, server2
HEAP SIZE: -Xms512m -Xmx512m -XX:MaxPermSize=256m
SERVER: PIA
HTTP: 8080
HTTPS: 8843 enabled
PID: 19098
RESOURCES: VSIZE=893.8m RSS=700.4m CPU=0.2%


The help message:


Usage 1: list process info for a app server and/or a prcs server
pl [-f] -c|p [-m -h] [-s]

Options:
-f force execution even appserv domain doesn't exsit
-c print processes of application server
-p print processes of process scheduler
-m print memory usage (virtual memory and RSS) and CPU usage(%)
-h print memory usage in human readable format
-s print summary and statistical info

Usage 2: calculate RAM/CPU usage of all app/prcs processes
pl -r

Usage 3: list setting and running info from a web domain (Weblogic only)
pl -w






Currently only Solaris version of the script is available. You can get it from here.


Update on 13-Oct-2009: Some bug fixes. pl -w produces more information.

Wednesday, May 6, 2009

Manipulating Child Rows in A PeopleSoft Component Through Web Services

I had this issue when trying to manipulate ID types for a user profile through web service. It was easy to update an existing ID type or to insert a non-existing ID type. For example, look at the following SOAP message:


<soapenv:Body>
<ns1:Update__CompIntfc__USER_PROFILE xmlns:ns1="http://xmlns.oracle.com/Enterprise/Tools/schemas/M274199.V1">
<ns1:UserID>COPYUSER2</ns1:UserID>
<ns1:IDTypes>
<ns1:IDType>EMP</ns1:IDType>
<ns1:Attributes>
<ns1:Fieldname>EmplID</ns1:Fieldname>
<ns1:Recname>PERSONAL_DATA</ns1:Recname>
<ns1:AttributeValue>AA0001</ns1:AttributeValue>
<ns1:AttributeName>EmplID</ns1:AttributeName>
</ns1:Attributes>
</ns1:IDTypes>
</ns1:Update__CompIntfc__USER_PROFILE>
</soapenv:Body>


If ID Type 'EMP' exists for user profile 'TESTUSER', system updates the ID Type's attribute value as 'AA0001'. Or if ID Type 'EMP' doesn't exist with user profile 'TESTUSER', it inserted.

However, this way won't work if we need to remove 'EMP' from 'TESTUSER'. We must adopt an attribute 'CINodeAction' to do the work:


<soapenv:Body>
<ns1:Update__CompIntfc__USER_PROFILE xmlns:ns1="http://xmlns.oracle.com/Enterprise/Tools/schemas/M274199.V1">
<ns1:UserID>TESTUSER</ns1:UserID>
<ns1:IDTypes CINodeAction="delete">
<ns1:IDType>EMP</ns1:IDType>
</ns1:IDTypes>
</ns1:Update__CompIntfc__USER_PROFILE>

As the matter of fact, we can also set value 'update' or 'insert' to 'CINodeAction' for first and second scenacios stated above, and this makes the SOAP message unambiguous and more understandable.

Attribute 'CINodeAction' is not documented, but can be digged from application package SOAPTOCI.

PS: This tip applies up to PeopleTools 8.49. In to-be-released PeppleTools 8.50, property 'action' has been announced together with some other properties.

Thursday, February 12, 2009

Badly-coded PeopleCode Fails Invocation of User-defined methods Through Web Services

I wrote a user-defined method in a PeopleSoft component interface (i.e. USER_PROFILE) which was exposed as a web services. But when I called this method from a web service client, I received below error:

The key UserID was not found in the request. (158,16017) PT_INTEGRATION.CIDefinition.OnExecute Name:setKeys PCPC:16560 Statement:306Called from:PT_INTEGRATION.CIDefinition.OnExecute Name:invokeUserDefinedFunction Statement:97Called from:PT_INTEGRATION.CIDefinition.OnExecute Name:OnEvent Statement:34

By using PeopleCode debugging, I traced the error to method setKeys of PT_INTEGRATION:CIDefinition and concluded it is the badly-coded PeopleCode that has resulted in the error.

Let's investigate the PeopleCode(partial) of method setKeys:


/* set the keys */
For &i = 1 To &ciKeyCollection.Count

&currentKey = &ciKeyCollection.item(&i);
&keyNotFound = True;

/* is there a corresponding element in the input XML? */

For &j = 1 To &rootNode.ChildNodeCount
/* NOTE: This is the bad code that causes the error! */
If (Upper(&rootNode.GetChildNode(&j).LocalName) = &currentKey.name) Then

/* the key is present in the XML */
&keyNotFound = False;

Local string &keyValue = &rootNode.GetChildNode(&j).NodeValue;

If (&keyValue = "") Then
throw CreateException(&ibMsgSetNumber, &emsgKeyValueNotInRequest, "No value found for the key %1 in the request.", &currentKey.name);
End-If;

/* set the key data */
&ciInstance.SetPropertyByName(&currentKey.name, &keyValue);

End-If;
End-For;

If (&keyNotFound) Then
throw CreateException(&ibMsgSetNumber, &emsgKeyNotInRequest, "The key %1 was not found in the request.", &currentKey.name);
End-If;

End-For;

The logic of method setKeys is clear: it tries to search the SOAP request for the component interface search key values and sets key values if the search succeeds, or throws an exception if it fails. For example, the search key of component interface USER_PROFILE 'UserId', method setKeys should be able to extract key value 'TESTUSER1' from SOAP request


<Method__CompIntfc__USER_PROFILE>
<UserID>TESTUSER1</UserID>
...
</Method__CompIntfc__USER_PROFILE>


However, things don't go as expected because of the following statement:


If (Upper(&rootNode.GetChildNode(&j).LocalName) = &currentKey.name) Then


This If-Then statement tries to compare a SOAP node with a search key name. You may have noticed the SOAP node name (&rootNode.GetChildNode(&j).LocalName) is formatted to upper case while the search key name (&currentKey.name) is not. It seems that the programmer assumed that the search key name was always upper case and so required no formatting, but how could he/she made such an arbitary assumption?

For component interface USER_PROFILE which I was working with, since the search key name happens to be 'UserId' instead of 'USERID', above comparison always gives a 'false' and the codes setting key values will never be executed, that is the reason why I saw error "The key %1 was not found in the request."

The resolution is simple, after modifying above statement as


If (Upper(&rootNode.GetChildNode(&j).LocalName) = Upper(&currentKey.name)) Then


the web service has executed properly.

Alternatively, the code can also be


If (&rootNode.GetChildNode(&j).LocalName = &currentKey.name) Then


but it is less safe obviously.

This code persists until PeopleTools rel 8.49.08. Hopefully Oracle will correct it in later release.

Tuesday, February 10, 2009

Scripts Listing Processes and Memory Usage of PeopleSoft Application Server and Process Scheduler

Sometimes I want to list all processes belonging to a PeopleSoft app/process domain. Although PSADMIN does provide some options for this purpose, none can satisfy me fully. For example:

./psadmin -c sstatus -d DOMAIN

- No JSH and WSH, also no PID (process id, which is important to me)

./psadmin -c pslist -d DOMAIN

- Shows PID, but no BBL, JSH and WSH

./psadmin -p status -d DBNAME

- No PID

Besides, I am also interested in the memory usage each process, but PSADMIN doesn't provide this kind of information too.

Therefore, I wrote 2 scripts in order to self-help. One is a shell script (view) runnable at Solaris. The other one is a VB script (view) runnable at Windows. A bat file (view) is provided as a wrapper to make the VB script run on and output to DOS commandline, it must reside at the same directory as the VB script.

Both scripts have similar syntax:

pl [-f] {-c-p-c -p} [-m [-h]] instance
Options: -f force execution even appserv domain doesn't exsit
      -c print processes of application server
      -p print processes of process scheduler
      -m print memory usage
      -h print memory usage in human readable format (only applicable to solaris version)


Sample output for running shell script:

$ ~/bin/pl -cmh DOMAIN
PID  PROCESS               VSIZE             RSS
---  -------               -----             ---
6431  PSSUBDSP             107.7M           92.5M
6430  PSPUBDSP             345.2M          161.5M
6397  BBL                   10.9M            8.9M
6407  PSSUBHND              99.7M           84.4M
6446  JREPSVR                8.9M            7.0M
6403  PSAPPSRV             514.7M          306.3M
8105  PSWATCHSRV            15.0M           11.9M
6429  PSBRKDSP             107.7M           92.5M
6402  PSAPPSRV             513.8M          297.5M
6442  JSL                   10.4M            8.4M
6401  PSAPPSRV             533.9M          317.6M
19680  PSMONITORSRV         102.4M           87.2M
6440  WSL                    9.4M            7.6M
6405  PSBRKHND             107.7M           92.5M
6406  PSPUBHND              99.9M           84.6M
6404  PSSAMSRV              97.4M           82.3M
6444  JSH                   12.8M           11.0M
6445  JSH                   16.0M           14.3M
6443  JSH                   16.0M           14.3M


Sample output for running VB script:
C:\WINDOWS\system32>pl -c -p -m DOMAIN
PID     Command         VSize   Working Set
---     -------         -----   -----------
5640    BBL             22.3M   5.8M
1764    PSAPPSRV        114.5M  43.1M
1692    PSAPPSRV        324.5M  81.3M
5856    PSAPPSRV        114.0M  42.5M
2448    PSSAMSRV        104.4M  40.4M
7060    PSANALYTICSRV   107.6M  41.0M
6556    PSANALYTICSRV   107.6M  41.0M
6364    PSANALYTICSRV   107.6M  41.0M
3348    PSDBGSRV        101.5M  37.3M
4592    PSRENSRV        79.4M   17.4M
8112    PSMONITORSRV    101.5M  37.6M
3984    WSL(7000)       24.3M   4.6M
4540    JSL(9000)       25.0M   4.6M
5452    JREPSVR         19.9M   3.9M
7292    PSSAMSRV        100.9M  37.2M
740     PSWATCHSRV      25.1M   6.7M

3840    WSH(7001)       23.9M   4.9M
820     JSH(9001)       24.0M   4.7M
6256    JSH(9002)       26.1M   5.5M
3164    JSH(9003)       24.0M   4.7M
6064    JSH(9004)       26.1M   5.0M
7732    JSH(9005)       24.0M   4.7M

PID     Command         VSize   Working Set
---     -------         -----   -----------
5552    BBL             21.7M   2.7M
5572    PSAESRV         109.9M  14.4M
5616    PSAESRV         109.9M  14.4M
5748    PSPRCSRV        130.8M  21.7M
7112    PSAESRV         303.4M  63.0M
7488    PSDSTSRV        282.0M  50.5M
5812    PSMONITORSRV    100.9M  37.6M


Update on 02/Oct/2009: the enhanced version of pl script (for solaris) is available here