====== Single pulse search tutorial ======

The goal of this page is to help the on-boarding of new students who are interesting in single pulse searches.

The tutorial uses ''PRESTO''. Although ''PRESTO'' may not be the best for a computationally intensive pipelines, it is pedagogically useful by allowing the user to manually run each pipeline step.

[[https://github.com/scottransom/presto/blob/master/docs/PRESTO_search_tutorial.pdf | Link to PRESTO tutorial in PRESTO's github repo]]

Notes:   
  * I providing the commands outside of singularity. Run them with singularity as you prefer. 
  * I assume that the input data is in filterbank format and done my best to note where commands may differ in the case of PSRFITS format. 

===== What you need =====

  - Data: short observation of a test pulsar (MY_DATA.fil)
  - Software: either a working ''PRESTO'' install or a singularity image containing ''PRESTO''

===== Simple SP pipeline =====

=== Step 1: inspect data ===

Become familiar with the structure and contents of the data using ''waterfaller.py''. 

Example command:

''> waterfaller.py %%--%%show-ts %%--%%show-spec %%--%%colour-map='viridis' -d 57 -T 100 -t 5 %%--%%downsamp=8 %%--%%scaleindep MY_DATA.fil''

Parameters to play around with:
  * Turn on and off dedispersion
  * Go to a different time in the file
  * Change the downsample factor
  * Look at data with and without independently scaling each frequency channel

=== Step 2: apply RFI mitigation ===

''PRESTOs'' removes RFI by calculating a mask. The mask informtion is saved in a number of files, one of which can be passed to the following steps. Therefore, a new, cleaned copy of the original data file is not produced. 

Example command:

''> rfifind -o mask_file_name -time 1.0 -zapchan N:N MY_DATA.fil''

Parameters to play around with:
  * Duration of processing time (-time)
  * In the case of PSRFITS data: -noscales -nooffsetts
  * Change thresholding parameters (-timesig, -freqsig, -chanfrac, -intfrac)
  * Add channels to manually zap (-zapchan)

Note, ''rfifind'' can be quite slow to run, but it is possible under some circumstances to recalculate the mask from the mask itself rather than rerunning the entire code.

Data file: MY_DATA.fil \\
Original mask: mask_file_name_rfifind.mask

''> rfifind -o mask_file_name -nocompute -freqsig 8 -mask mask_file_name_rfifind.mask MY_DATA.fil''

=== Step 3: generate dedispersed time series ===

This command assumes the data are for B0355+54 - change the DM range accordingly if using a different test pulsar. 

This command can be used to generate a single dedipsersed time series by setting -numdms 1. Having dedispered time series over a range around the true value is useful for in Step 4. 

''> prepsubband -nobary -lodm 52.4 -dmstep 0.5 -numdms 20 -clip 0 -downsamp 1 -mask mask_file_name_rfifind.mask -o my_ts MY_DATA.fil''

Variations:
  * PSRFITS files: if you are searching a PSRFITS file and applied ''%%--%%noscales %%-nooffsets%%'' in the ''RFIfind'' stage, you should do so here too.
  * Change the DM step and DM range (and see its impact on Step 4)
  * Change the downsampling factor
  * Run with clipping to see if this changes the time series

Manually inspect the time series:

''> exploredat my_ts.dat''

  * Can you see bright pulses?
  * Note the slowly varying
  * If you've applied an RFIfind mask, what impact does this have on the time series?

=== Step 4: generate single pulse candidates ===

''> single_pulse_search.py -m 30 -t 6 -b *.dat''

Parameters to vary:
  * Turn on bad block flagging
  * Use different detrend chunksizes 

Look at SP diagnostic plot: my_ts_singlepulse.ps

Recreate this plot with a different threshold or time range:

''> single_pulse_search.py -m 30 -t 8 -s 0 -e 120 -b *.singlepulse''

=== Step 5: machine learning classifier ===

To be done...