Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

360 - Refactor Dataset Page to handle dataset ID via URL query #364

29 changes: 14 additions & 15 deletions src/components/Dataset/DatasetPageHeader.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
/* eslint-disable no-nested-ternary */
import moment from 'moment'
import { Box, Heading } from 'grommet'
import { usePageRendered } from 'hooks/usePageRendered'
Expand Down Expand Up @@ -35,16 +34,18 @@ const Block = ({ children }) => {
export const DatasetPageHeader = ({ dataset }) => {
const pageRendered = usePageRendered()
const { setResponsive } = useResponsive()
const {
expires_on: expiredOn,
is_available: isAvailable,
is_processed: isProcessed,
is_processing: isProcessing,
success
} = dataset
const isExpired = moment(expiredOn).isBefore(Date.now())
const isProcessingError = success === false // 'success' may be null
Comment on lines +37 to +45
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should replace this with a helper that wraps the actual meaning of these state attributes. They are not the clearest names on the API. We can't really change them very easily there because external scripts might be using them so here we should just abstract away the specifics into the actionable states.

I think you can clean this up but the important part is that logically only one of these states can be true at one #time.

  const getDatasetState = (dataset) => {

    // datasets go in the following states:
    // isNotProcessed -> isProcessing -> isReady -> isExpired
    //                              | -> isError


    // processing was attempted but an error occurred and no output was able to be created
    const isErrored = dataset.success === false

    // the dataset is still available to be edited by the token holder
    const isNotStarted = dataset.is_processing === false && dataset.is_processed === false && dataset.success === null

    // dataset is processing and has not encountered an error
    const isProcessing = dataset.is_processing && dataset.is_processed === false && dataset.success === null

    // processing completed but is no longer available
    const isExpired = dataset.is_processed && dataset.success === true && moment(dataset.expires_on).isBefore(Date.now())

    // dataset is ready to download and has not expired
    const isReady = dataset.is_processed && !isExpired


    return {
      isNotStarted,
      isProcessing,
      isReady,
      isExpired,
      isErorred,
    }
  }

Copy link
Member Author

@nozomione nozomione Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

David, as chat on Slack (and noted in here as well #364 (comment)), I've filed a dedicated new issue (#370), for adding getDatasetState helper and refactoring the components that use dataset states.

This should make the review easier while keeping the file changes minimal and reducing the merge conflicts (the PR for it is also ready (#373). Thank you David!


if (!pageRendered) return null

const expiredOn = dataset?.expires_on
const isAvailable = dataset?.is_available
const isExpired = moment(expiredOn).isBefore(Date.now())
const isProcessed = dataset?.is_processed
const isProcessing = dataset?.is_processing
const isProcessingError = dataset?.success === false // 'success' may be null

if (isProcessingError) {
return (
<Block>
Expand Down Expand Up @@ -79,13 +80,11 @@ export const DatasetPageHeader = ({ dataset }) => {
return (
<FixedContainer pad="none">
<Box>
{dataset?.data && (
<Box pad={{ top: 'large', bottom: 'medium' }}>
<Heading level={2} size={setResponsive('small', 'large')}>
Shared Dataset
</Heading>
</Box>
)}
<Box pad={{ top: 'large', bottom: 'medium' }}>
<Heading level={2} size={setResponsive('small', 'large')}>
Shared Dataset
</Heading>
</Box>
</Box>
</FixedContainer>
)
Expand Down
19 changes: 11 additions & 8 deletions src/components/Dataset/MoveToDatasetButton.js
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you give me feedback on this suggestion?
I think I'd like to make the following changes for consistency.

  • Keep the prop as dataset since that is the dynamic part and would match convention.
  • Instead of renaming dataset.data to datasetData always use dataset.data
  • Instead of renaming dataset.data and dataset.id from to myDatasetid and myDatasetData just rename the whole dataset to myDataset so we will always access attributes on myDataset or dataset.

In the future we may consider makeing myDataset an export of useDatasetManager but I need to review that holistically before suggesting that change.

Copy link
Member Author

@nozomione nozomione Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for these great suggestions 👍 I totally agree with keeping the prop as dataset and using dataset.data. Renaming the dataset context state to myDataset is an excellent idea for clarity and readability as well! Let’s discuss this further in our 1:1, and once we finalize the implementation strategy, we can file a new issue for it. Looking forward!

Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,30 @@ import { Button } from 'components/shared/Button'
import { Modal } from 'components/shared/Modal'
import { MoveToDatasetModal } from './MoveToDatasetModal'

export const MoveToDatasetButton = ({ dataset, disabled, selectedDataset }) => {
export const MoveToDatasetButton = ({ dataset }) => {
const { push } = useRouter()
const { addSamples, getTotalSamples } = useDatasetManager()
const {
dataset: { id: myDatasetId, data: myDatasetData },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dataset: { id: myDatasetId, data: myDatasetData },
dataset: myDataset,

And then access the attributes in the same way so that it is consistent.

addSamples,
getTotalSamples
} = useDatasetManager()
const { openModal, closeModal } = useModal()
const id = `move-to-dataset-${dataset?.id}`
const id = `move-to-dataset-${myDatasetId}`
const radioOptions = [
{ label: 'Append samples to My Dataset', value: 'append' },
{ label: 'Replace samples in My Dataset', value: 'replace' }
]
const defaultValue = radioOptions[0].value
const [value, setValue] = useState(defaultValue)
const newTotalSamples = getTotalSamples(selectedDataset.data)
const totalSamples = getTotalSamples(dataset.data)
const newTotalSamples = getTotalSamples(dataset.data)
const totalSamples = getTotalSamples(myDatasetData)
const pathname = '/download'

const handleMoveToDataset = async () => {
if (totalSamples > 0) {
openModal(id)
} else {
await addSamples(selectedDataset.data)
await addSamples(dataset.data)
push(
{
pathname,
Expand All @@ -47,7 +51,6 @@ export const MoveToDatasetButton = ({ dataset, disabled, selectedDataset }) => {
id={id}
button={
<Button
disabled={disabled}
label="Move to Dataset"
secondary
responsive
Expand All @@ -61,7 +64,7 @@ export const MoveToDatasetButton = ({ dataset, disabled, selectedDataset }) => {
id={id}
closeModal={closeModal}
defaultValue={defaultValue}
dataset={selectedDataset}
dataset={dataset}
pathname={pathname}
radioOptions={radioOptions}
value={value}
Expand Down
5 changes: 2 additions & 3 deletions src/components/Dataset/ShareDatasetButton.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,8 @@ export const ShareDatasetButton = ({ dataset }) => {
}, 3000)
const [isCopied, setIsCopied] = useState(false)
const [value, copyText] = useCopyToClipboard(null)
const { id: datasetId } = dataset
const id = `shareable-link_${datasetId}`
const shareableLink = `${getDomain()}/dataset/${datasetId}?ref=share`
const id = `shareable-link_${dataset.id}`
const shareableLink = `${getDomain()}/dataset/${dataset.id}?ref=share`

const onShareClick = () => {
openModal(id)
Expand Down
3 changes: 1 addition & 2 deletions src/components/Download/DatasetSummary/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ const getExperimentCountBySpecies = (data, experiments) => {
export const DatasetSummary = ({ dataset }) => {
const { getTotalExperiments, getTotalSamples } = useDatasetManager()
const { setResponsive } = useResponsive()
const samplesBySpecies = dataset.organism_samples
const totalSamples = getTotalSamples(dataset.data)
const totalExperiments = getTotalExperiments(dataset.data)
const experimentCountBySpecies = getExperimentCountBySpecies(
Expand Down Expand Up @@ -71,7 +70,7 @@ export const DatasetSummary = ({ dataset }) => {
</TableHeader>
<TableBody style={{ fontSize: setResponsive('16px', '18px') }}>
<SpiecesRow
samplesBySpecies={samplesBySpecies}
samplesBySpecies={dataset.organism_samples}
experimentCountBySpecies={experimentCountBySpecies}
/>
<TotalRow totals={[totalSamples, totalExperiments]} />
Expand Down
9 changes: 9 additions & 0 deletions src/components/Download/DownloadDatasetButton.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,17 @@ export const DownloadDatasetButton = ({
dataset,
label = 'Download Dataset'
}) => {
const {
is_processing: isProcessing,
is_processed: isProcessed,
success
} = dataset
const { openModal, closeModal } = useModal()
const id = `download-dataset_${dataset.id}`
const isUnprocessedDataset = // sets visibility of the Download Dataset button
!isProcessing && !isProcessed && success !== false

if (!isUnprocessedDataset) return null

return (
<Modal
Expand Down
32 changes: 21 additions & 11 deletions src/components/Download/FilesSummary.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,22 @@ const Card = ({ description, format, index, title }) => {
)
}

export const FilesSummary = ({ dataset, defaultDataset = {}, isProcessed }) => {
export const FilesSummary = ({ dataset }) => {
const {
organism_samples: organismSamples,
aggregate_by: aggregateBy,
scale_by: scaleBy,
quantile_normalize: quantileNormalize,
is_processed: isProcessed,
success
} = dataset
const { createDataset, updateDataset, regeneratedDataset } =
useDatasetManager()
const { setResponsive } = useResponsive()
const isDownloadOptions = isProcessed && success // sets visibility of the download options

// returns the file size estimates of the given dataset and its aggregate_by value (either 'EXPERIMENT' or 'SPECIES')
const downloadFilesData = (data, samplesBySpecies, aggregateBy) => {
const downloadFilesData = (data, samplesBySpecies, aggregateByOption) => {
const totalExperiments = Object.keys(data).length
// metadata info of a download https://github.com/AlexsLemonade/refinebio-frontend/issues/25#issuecomment-395870627
// the samples aggregated by 'EXPERIMENT'
Expand Down Expand Up @@ -91,16 +101,15 @@ export const FilesSummary = ({ dataset, defaultDataset = {}, isProcessed }) => {
}
}

return aggregateBy === 'SPECIES'
return aggregateByOption === 'SPECIES'
? aggregatedBySpecies(dataset, samplesBySpecies)
: aggregatedByExperiment(dataset)
}

const samplesBySpecies = dataset.organism_samples
const fileSummaries = downloadFilesData(
dataset.data,
samplesBySpecies,
dataset.aggregate_by
organismSamples,
aggregateBy
)
const transformationOptions = options.transformation.reduce(
(acc, cur) => ({ ...acc, [cur.value]: cur.label }),
Expand All @@ -116,6 +125,7 @@ export const FilesSummary = ({ dataset, defaultDataset = {}, isProcessed }) => {
params
)
const pathname = `/dataset/${response.id}`
const defaultDataset = {} // TODO: This will be cleaned up in Issue #359

gtag.trackRegeneratedDataset(
defaultDataset,
Expand All @@ -133,17 +143,17 @@ export const FilesSummary = ({ dataset, defaultDataset = {}, isProcessed }) => {
Download Files Summary
</Heading>

{isProcessed && (
{isDownloadOptions && (
<Box margin={{ bottom: 'small' }}>
{!openForm && (
<Box direction="row" gap="xlarge">
<Text weight="bold">
Aggregate by: {formatString(dataset.aggregate_by)}
Aggregate by: {formatString(aggregateBy)}
</Text>
<Text weight="bold">
Transformation: {transformationOptions[dataset.scale_by]}
Transformation: {transformationOptions[scaleBy]}
</Text>
{!dataset.quantile_normalize && (
{!quantileNormalize && (
<Text weight="bold">
Quantile Normalization Skipped for RNA-seq samples
</Text>
Expand All @@ -162,7 +172,7 @@ export const FilesSummary = ({ dataset, defaultDataset = {}, isProcessed }) => {
<DownloadOptionsForm
dataset={dataset}
buttonLabel="Regenerate Dataset"
isProcessed={isProcessed}
isProcessed={isDownloadOptions}
onSubmit={handleRegenerateDataset}
/>
</ExpandableBlock>
Expand Down
112 changes: 51 additions & 61 deletions src/pages/dataset/[dataset_id].js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { useEffect, useState } from 'react'
import { useRouter } from 'next/router'
import { Box } from 'grommet'
import { useDatasetManager } from 'hooks/useDatasetManager'
import { usePageRendered } from 'hooks/usePageRendered'
Expand All @@ -25,48 +26,48 @@ export const getServerSideProps = ({ query }) => {
return { props: { query } }
}

export const Dataset = ({ query }) => {
export const Dataset = ({ query: { dataset_id: datasetId, start } }) => {
const pageRendered = usePageRendered()
const { push } = useRouter()
const { setResponsive } = useResponsive()
const { dataset_id: idFromQuery, start } = query
const { dataset, datasetId, error, loading, getDataset, regeneratedDataset } =
useDatasetManager()
const {
datasetId: myDatasetId,
error,
loading,
getDataset
} = useDatasetManager()
const { isProcessingDataset, polledDatasetState, pollDatasetId } =
usePollDatasetStatus()
const [selectedDataset, setSelectedDataset] = useState({}) // stores the dataset currently displayed on the page
const isProcessed = selectedDataset?.is_processed && selectedDataset?.success // sets visibility of the download options in Dwonload Files Summary
const isUnprocessedDataset = // sets visibility of the Download Dataset button
!selectedDataset?.is_processing &&
!selectedDataset?.is_processed &&
selectedDataset?.success !== false
const [dataset, setDataset] = useState({}) // dataset currently displayed on the page

const getSelectedDataset = async (id) => {
const getDatasetFromQuery = async (id) => {
const response = await getDataset(id)
setSelectedDataset(response)
setDataset(response)
}

useEffect(() => {
getSelectedDataset(idFromQuery)
pollDatasetId(idFromQuery) // sets a processing datasets for polling
}, [query])
// redirects users to /download if datasetId matches My dataset ID
if (datasetId === myDatasetId) push('/download')

getDatasetFromQuery(datasetId)
pollDatasetId(datasetId) // sets a processing datasets for polling
}, [datasetId])

useEffect(() => {
// swaps selectedDataset to the last fetched polledDatasetState
// swaps dataset to the last fetched polledDatasetState
// (only if the dataset ID in URL was being processed) to update
// DatasetPageHeader
if (!isProcessingDataset && polledDatasetState) {
setSelectedDataset(polledDatasetState)
setDataset(polledDatasetState)
}
}, [isProcessingDataset, polledDatasetState])

if (!pageRendered) return null

const isSameId = datasetId === idFromQuery

if (start) {
return (
<FixedContainer>
<StartProcessing dataset={selectedDataset} />
<StartProcessing dataset={dataset} />
</FixedContainer>
)
}
Expand All @@ -82,52 +83,41 @@ export const Dataset = ({ query }) => {
)
}

if (loading) {
return (
<Box align="center" fill justify="center" margin={{ top: 'large' }}>
<Spinner />
</Box>
)
}

return (
<FixedContainer>
{loading ? (
<Box align="center" fill justify="center" margin={{ top: 'large' }}>
<Spinner />
</Box>
) : (
<Box>
{selectedDataset?.data && (
<>
<DatasetPageHeader dataset={selectedDataset} />
<Box>
{dataset?.data && (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can get rid of this check as well there should only be 2 states we care about. Loading and loaded. I would think that if dataset.data is undefined then loading would be true.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been cleaned up in 41e8260 👍

<>
<DatasetPageHeader dataset={dataset} />
<Row
border={{ side: 'bottom' }}
pad={{ bottom: setResponsive('medium', 'small') }}
>
<Box>
<MoveToDatasetButton dataset={dataset} />
</Box>
<Row
border={{ side: 'bottom' }}
pad={{ bottom: setResponsive('medium', 'small') }}
gap={setResponsive('medium', 'small')}
margin={{ top: setResponsive('medium', 'none') }}
>
<Box>
<MoveToDatasetButton
dataset={dataset}
selectedDataset={selectedDataset}
disabled={isSameId}
/>
</Box>
<Row
gap={setResponsive('medium', 'small')}
margin={{ top: setResponsive('medium', 'none') }}
>
<ShareDatasetButton dataset={selectedDataset} />
{isUnprocessedDataset && (
<DownloadDatasetButton dataset={selectedDataset} />
)}
</Row>
<ShareDatasetButton dataset={dataset} />
<DownloadDatasetButton dataset={dataset} />
</Row>
<FilesSummary
dataset={regeneratedDataset || selectedDataset}
defaultDataset={selectedDataset}
isProcessed={isProcessed}
/>
<DatasetSummary dataset={regeneratedDataset || selectedDataset} />
<DatasetDetails
dataset={regeneratedDataset || selectedDataset}
isImmutable
/>
</>
)}
</Box>
)}
</Row>
<FilesSummary dataset={dataset} />
<DatasetSummary dataset={dataset} />
<DatasetDetails dataset={dataset} isImmutable />
</>
)}
</Box>
</FixedContainer>
)
}
Expand Down