Step 1 - Get a computer account

To access and process your LCLS experiment data, you first need a valid SLAC UNIX account, enabled in the LCLS System. To this end, you will need to follow the steps below (and for more detailed information, check out this Confluence documentation page). The specific process depends on your role at SLAC.

For LCLS Users

Apply for a UNIX account through the User Portal.
Take the course CS101 - cyber security basics.
Understand the Acceptable Use of SLAC Information Technology Resources.
The experiment spokesperson adds your UNIX account to the Experiment Manager.

For Interns and Collaborators

Fill out the Site Access Request Form to request a SLAC ID number. Non citizens will need to provide additional information including passport photos and the I-94 form available from the U.S. Customs and Boarder Protection webpage. Note for host (POC): you may need to complete additional training too to host non citizens -- ask VUE for more details.
Your SLAC point of contact (POC) will receive an email containing the SLAC ID Number. Note that the process will take slightly longer for non US citizens.
Follow the "Manual Login" instructions to get a password and log into the Web training portal.
Complete ESH219 and CS200 as minimal training needed to obtain badge.
Your point of contact will request WINDOWS and UNIX accounts for you using the SLAC Computer Account Request form. More info can be found about the process here. Note for POC: you need to fill the form with your own name under "Requested for", but put in the correct info in other fields.
Day 1 onsite, obtain your badge (you will need POC signature on the paper form there) at building 53.
For Stanford students joining as RA, additional forms are needed by your Stanford departments.
If you will be using NERSC, go through this instruction to create account and let POC know to add you to the correct project.

Enabling access to the S3DF

The SLAC-wide Shared Science Data Facility (S3DF) will soon replace the computer and storage resources managed by LCLS Photon Controls and Data Systems (PCDS) currently used for LCLS experiment data management. In order to use S3DF, it is necessary to enable your S3DF account using your SLAC UNIX account. Detailed information in the S3DF documentation here.

Step 2 - Understand LCLS Data Systems

Now that you can access the computer systems at SLAC, it is time to get yourself oriented. For that, it is useful to understand how the data flows from the beamline to storage, a.k.a to understand the LCLS Data Systems, built with 3 successive levels:

Level #	Processing Time	Storage Duration	Access
1. Data Acquisition and Reduction	diagnostics within <1s.	None. Data is in shared memory.	Experts only
2. Fast Feedback	diagnostics and results within <1min.	<1 week.	(before) - through `pslogin` to `psffb`. (now) - on S3DF.
3. Offline Analysis	intensive compute during and after the beamtime.	<4 months before being moved to tape.	(before) - through `pslogin` to `psana`. (now) - on S3DF

The figure below provides a high-level map of the LCLS Data Systems presenting the main entities and their networks. (more TBD).


LCLS Data Systems. TBD.

migration to S3DF

S3DF is a new capability at SLAC and is currently still a work in progress. New experiments will be stored and processed on S3DF while previous experiments can still be access on the PCDS clusters. On S3DF, it will be more seamless to switch from online analysis to offline analysis between shifts and at the end of the beamtime.

Step 3 - Access your experiment data

Your experiment data cannot be reached directly from external networks. You must first access a login node.


Pathways to the Experiment. TBD.

The exact pathway will depend on whether your experiment happened before 2023 or not and whether you intend to do online analyis on the FFB nodes or offline analysis. We detail these different scenarii below.

Recent experiments (2023 - ...)

It is anticipated that experiments carried out in 2023 or after will be processed directly on S3DF. First, get in the system:

ssh <username>@s3dflogin.slac.stanford.edu

In order to get access to the data and to submit SLURM jobs, reach an interactive node:

ssh psana

Once on the interactive node, execute the following command (or place it in your ~/.bashrc) in order to set the necessary environment variables and environments that will allow you to find the data and process it:

source /sdf/group/lcls/ds/ana/sw/conda1/manage/bin/psconda.sh

FFB at S3DF

In S3DF, both FFB and Offline storage are accessible. By default, the command above points to the offline storage. In order to access the FFB storage, you can override the environment variable that points to the data: export SIT_PSDM_DATA=/sdf/data/lcls/drpsrcf/ffb

The experiments and their data can be found at the following path:

${SIT_PSDM_DATA}/<instrument>/<experiment>

More S3DF info

For more information about the S3DF, check out the PCDS documentation here and the official S3DF docs here.

Past experiments (pre 2023)

First, get in the system:

ssh <username>@pslogin.slac.stanford.edu

From there, you can access the FFB nodes or offline analysis nodes:

ssh psana # offline nodes
# or 
ssh psffb # FFB nodes

In order to set the necessary environment to start processing it, execute the following command:

source /reg/g/psdm/etc/psconda.sh

The experiments and their data can be found at the following path:

/cds/data/psdm/<instrument>/<experiment>     # Offline storage
/cds/data/drpsrcf/<instrument>/<experiment>  # FFB storage

Data import/export

Note that interactive nodes are not exposed to external networks. If you need to transfer data, you can go through psexport.slac.stanford.edu.

More FFB info

For more information about the Fast Feedback System, check out the PCDS documentation here.