Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Gibbs sampler bug and add initial_amplitude parameter, and fix misc. MPIArray warnings #207

Merged
merged 5 commits into from
Nov 18, 2022

Conversation

sjforeman
Copy link
Contributor

@sjforeman sjforeman commented Oct 24, 2022

This PR has turned into a Frankenstein of various edits, but they are all relatively minor:

  • Fix a few MPIArray warnings I've found in various tasks.

  • Fix the following bug in DelaySpectrumEstimatorBase: if the input container has a large number of "baselines" with zero data or weights, such that there is an MPI rank that only contains these baselines, the routine that initializes the random number generator will never be called on this rank, because self.rng is only called in the argument of delay_spectrum_gibbs() in the main loop:

    for lbi, bi in delay_spec.spectrum[:].enumerate(axis=0):
    self.log.debug(f"Delay transforming baseline {bi}/{nbase}")
    # Get the local selections
    data = data_view[lbi].view(np.ndarray)
    weight = weight_view[lbi].view(np.ndarray)
    # Mask out data with completely zero'd weights and generate time
    # averaged weights
    weight_cut = (
    1e-4 * weight.mean()
    ) # Use approx threshold to ignore small weights
    data = data * (weight > weight_cut)
    weight = np.mean(weight, axis=0)
    if (data == 0.0).all():
    continue
    # If there are no non-zero weighted entries skip
    non_zero = weight > 0
    if not non_zero.any():
    continue
    # Remove any frequency channel which is entirely zero, this is just to
    # reduce the computational cost, it should make no difference to the result
    data = data[:, non_zero]
    weight = weight[non_zero]
    non_zero_channel = channel_ind[non_zero]
    spec = delay_spectrum_gibbs(
    data,
    ndelay,
    weight,
    initial_S,
    window=self.window if self.apply_window else None,
    fsel=non_zero_channel,
    niter=self.nsamp,
    rng=self.rng,
    complex_timedomain=self.complex_timedomain,
    )
    # Take an average over the last half of the delay spectrum samples
    # (presuming that removes the burn-in)
    spec_av = np.median(spec[-(self.nsamp // 2) :], axis=0)
    delay_spec.spectrum[bi] = np.fft.fftshift(spec_av)
    , and this will always be skipped. The solution is to explicitly initialize the RNG on all ranks before entering the loop, by calling rng = self.rng.

  • Allow the user to specify the amplitude of the initial power spectrum guess in DelaySpectrumEstimator and DelaySpectrumEstimatorBase.

@sjforeman sjforeman changed the title Sf/mpiarray warnings Fix Gibbs sampler bug and add initial_amplitude parameter, and fix misc. MPIArray warnings Nov 15, 2022
@sjforeman sjforeman merged commit fd0153a into master Nov 18, 2022
@sjforeman sjforeman deleted the sf/mpiarray-warnings branch November 18, 2022 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants