FTP transfers as PeopleSoft Processes

There are various ways to initiate FTP file transfers in a PeopleSoft environment, e.g. via PeopleCode, UNIX shell scripts, etc. However, the most flexible and integrated option is to define a FTP transfer as a PeopleSoft Process. This approach allows you to use FTP transfers as part of Jobs defined and scheduled in PS Process Scheduler. FTP transfers will integrate into PS nicely, allowing you to:

• Have one unified way of FTP transfers, regardless which application (SQR, Application Engine, etc.) produced or receives the file
• Schedule FTP transfers in Process Scheduler
• Make other PS processes depending on the state of an FTP transfer
• Monitor and access logfiles of FTP transfers in Process Monitor and Report Manager
• Migrate FTP transfer definitions as PeopleSoft objects

Creation of FTP Process Type

In order to allow the definition of FTP transfers as PS Processes, you first have to define a PS Process Type for your environment. This can be done in " Process Scheduler Manager" in the PeopleTools menu.

Please note that the definition of a Process Type is per Operating System and Database Type.

In this example we will define FTP transfers for a Process Scheduler running in UNIX / Oracle environment.

'Generic Process Type' is irrelevant and only determines what type of files (by extension) the PS Distribution Agent will pick up for this Process Type. Use 'AppEngine' to ensure that your logfiles will be handled properly by Distribution Agent.

Process Scheduler will invoke whatever you have specified in the 'Command Line', i.e. it will invoke 'prcs_ftp' rather than any AppEngine.

'Parameter List' reads:

%%PRCSNAME%% %%ACCESSID%% %%ACCESSPSWD%% %%RUNCNTLID%% %%INSTANCE%% 
%%DBNAME%%

Sample FTP transfer Process

Based on the FTP Process Type we can now define individual FTP transfers, passing the FTP commands as parameters.

'Parameter List' will contain the FTP commands to execute. Those parameters will be sent to the procs_ftp shell script defined in the underlying UNIX FTP Process Type.

Syntax and order of these parameters is very important. They have to match what the prcs_ftp script is expecting. If you need to modify the prcs_ftp shell script, ensure that all your Process Definition conforms to those changes implemented in prcs_ftp.

For our sample FTPDEMO Process the parameters are:

;ftp.agentbob.info;userid;;get;RemoteFilename.txt;LocalFilename.txt;;N

In case of a put command the order of remote and local file are reversed, i.e.

;ftp.agentbob.info;userid;;put;LocalFilename.txt;RemoteFilename.txt;;N

ftp.agentbob.info

(blank)

get

RemoteFilename.txt

LocalFilename.txt

(blank)

N

Configuring Process Scheduler to execute FTP transfers

Process Scheduler will only execute Process Types defined to be executable in its environment based on Operating System and Database Type. Therefore we will need to tell it that it should execute FTP transfers in Process Scheduler Manager -> Use -> Server Definitions. Just insert a new line, selecting 'Unix FTP' as Process Type and specify 'Priority' and 'Max Concurrent' as appropriate for your environment. See below:

Passwords for FTP user IDs & encryption passwords

We don't define the password for an FTP user ID with the other parameters in the FTP Process definition for security and manageability reasons. If the password was defined with the parameters in clear-text:

• Everybody developer having access to the FTP Process definition could read it.
• In a production environment corporate policies require passwords have to be changed regularly, usually every few months. We don't want to change the process definition every few months though.

Storing passwords in an encrypted form outside of the PS Process definition seems the most attractive option. We do this on a per PS system basis so that we can maintain passwords separately for different PS system running on the same server. In this demo the UNIX crypt utility is used to encrypt passwords

If you require a decent encryption, replace UNIX crypt with your choice of encryption utility, I am only using crypt for simplicity here. Do not rely on the standard UNIX crypt to provide any particular strength of encryption or protection.

Password and encryption keys are stored in the following directory structure:

/path/to/pshome/                                 (<-- path to your PS installation,
                                                      i.e. this is $PS_HOME)
             .PTDEV/                             (<-- PTDEV is the database name
                                                      of the PS system)
                 .userid@ftp.agentbob.info.crypt (<-- encrypted password for
                                                      userid on ftp.agentbob.info)
                 .userid@ftp.agentbob.info.key   (<-- encryption key for all transfers
                                                      using userid on ftp.agentbob.info)
             .PTSYS/                             (<-- PTSYS is the database name
                                                      of the PS system)
                 .userid@ftp.agentbob.info.crypt (<-- encrypted password for
                                                      userid on ftp.agentbob.info)
                 .userid@ftp.agentbob.info.key   (<-- encryption key for all transfers
                                                      using userid on ftp.agentbob.info)

If an FTP Process is run on PTDEV, it would take the password for userid on ftp.agentbob.info from the file /path/to/pshome/.PTDEV/.userid@ftp.agentbob.info.crypt. The same FTP Process run on PTSYS would use password stored in /path/to/pshome/.PTSYS/.userid@ftp.agentbob.info.crypt.

UNIX FTP script

Download the actual UNIX script (text/plain, 9.7 kB, info) and save it as /path/to/pshome/bin/prcs_ftp, make sure it is executable (chmod +x prcs_ftp). Ensure that $ORACLE_HOME and $PS_HOME are properly set when the script is executed.

There a few occurrences of /path/to/pshome in this script. Replace them with the appropriate path for your installation. Unfortunately some of them are in the FTP commands, so they cannot be easily replaced with the environment variable $PS_HOME. Something I will fix at some point......

Location of FTPed files

All files to be sent are expected to be found in /path/to/pshome/appserv/prcs/$DBNAME/files. Similarly, all files received will be stored in /path/to/pshome/appserv/prcs/$DBNAME/files. This is the default directory for all PS Processes, so FTPed files can easily be written or picked up by Application Engines or SQRs.

Archives of FTPed files

In addition to reading and writing files used in an FTP Process to and from /path/to/pshome/appserv/prcs/$DBNAME/files, a GZIPped copy of all files sent and received is stored in /path/to/pshome/appserv/prcs/$DBNAME/files/archive. Date/time will be automatically appended to the file name to allow multiple versions of the same files name.

Log and Trace files

Each FTP Process will produce log and trace files which will be distributed by the PS Distribution Agent according to the options defined for the FTP Process in PeopleSoft.