Cross-exposure implementation in picca

Corentin Ravoux - LPCA, Clermont-Ferrand

with Michael and Eric

 

23 April 2024

 

Reminder on cross-exposure

.

.

.

Standard

Cross-exposure

.

.

.

QSO 1

QSO i

QSO N

QSO 1

.

.

.

.

.

.

.

.

.

.

.

.

QSO N

Exposure Nexp

Exposure k

Exposure 1

\delta_{1}
\delta_{i}
\delta_{N}
\delta_{1,1}
\delta_{1,m}
\delta_{1,N_{exp}}

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

After coadding

Before coadding

Standard

Cross-exposure

.

.

.

.

.

.

QSO 1

QSO i

QSO N

QSO 1

.

.

.

.

.

.

.

.

.

.

.

.

QSO N

Exposure Nexp

Exposure k

Exposure 1

\delta_{1}
\delta_{i}
\delta_{N}
\delta_{1,1}
\delta_{1,m}
\delta_{1,N_{exp}}

Reminder on cross-exposure

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

Standard

Cross-exposure

P_{1\mathrm{D},\alpha}(k) = \left\langle P_{\mathrm{raw},i}(k) \right\rangle_{i\in[1...N]}
P_{\mathrm{raw},i}(k) = \left|\delta_{i}(k)\right|^2

.

.

.

.

.

.

QSO 1

QSO i

QSO N

QSO 1

.

.

.

.

.

.

.

.

.

.

.

.

QSO N

Exposure Nexp

Exposure k

Exposure 1

\delta_{1}
\delta_{i}
\delta_{N}
\delta_{1,1}
\delta_{1,m}
\delta_{1,N_{exp}}

Reminder on cross-exposure

P_{\mathrm{raw},i}(k) = \frac{1}{N_{\mathrm{exp}}(N_{\mathrm{exp}} - 1)} \sum_m \sum_{n\neq m} \left| \delta_{i,m}(k) \delta_{i,n}^{*}(k) \right|

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

In practice

Taking all exposure instead of coadds

 
spectra-main-dark-*.fits.gz

instead of

coadd-main-dark-*.fits.gz
f_{i,m}(\lambda)

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

In practice

Taking all exposure instead of coadds

 

Continuum fitting for all the exposures independently

Fourrier transform of deltas of each exposures

f_{i,m}(\lambda)
\delta_{i,m}(\lambda)
\delta_{i,m}(k)

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

In practice

Taking all exposure instead of coadds

 

Continuum fitting for all the exposures independently

Fourrier transform of deltas of each exposures

Combining all exposures with cross-exposure estimator

f_{i,m}(\lambda)
\delta_{i,m}(\lambda)
\delta_{i,m}(k)
P_{\mathrm{raw},i}(k) = \frac{1}{N_{\mathrm{exp}}(N_{\mathrm{exp}} - 1)} \sum_m \sum_{n\neq m} \left| \delta_{i,m}(k) \delta_{i,n}^{*}(k) \right|

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

In practice

Taking all exposure instead of coadds

 

Continuum fitting for all the exposures independently

Fourrier transform of deltas of each exposures

Computing average P1D

f_{i,m}(\lambda)
\delta_{i,m}(\lambda)
\delta_{i,m}(k)
P_{1\mathrm{D},\alpha}(k)

Combining all exposures with cross-exposure estimator

P_{\mathrm{raw},i}(k)

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

Previous work

  • First test with a very simple modification to picca: change all LOSID to "LOSID_EXPID"

 

  • Not possible to mask DLAs

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

Pull Request in picca

  • Large pull request which modifies delta extraction and p1d part of picca

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

Pull Request in picca

  • Modification in the reading of spectra, with a specific option
  • Keeping standard LOS_ID for the masking part (DLAs, lines)
  • Special treatment of the continuum fitting, dedicated to the multi exposure case
  • Implementation of the cross-exposure estimator in the P1D part
  • New format of individual P1D

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

Open question

  • How do we perform continuum fitting for cross-exposures ?

Pros

Cons

For independent exposures

For the best exposure of a quasar

On the coadd, applied on single exposures

  • Easy to implement
  • Relatively fast
  • Same quasar with different continuum
  • Loss of low SNR exposures
  • Long on large datasets
  • Loss of quasars with only low exposures.
  • Minimal loss of low SNR exposure quasar
  • Not in adequacy with picca or way too long on Y1
  • Easy to implement

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April

Next steps

  • Finish tests of PR (unitary test + Y1 results)
  • Measure sidebands / cross-tile / cross-exp within a night (check correlated noise)
  • Test on pixel level simulations
  • Issue: super slow reading (up to 6-7 hours for Y1)
  • New results in progress..

Corentin Ravoux                                                                                                                                                              Lyman-alpha meeting - 23 April