Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cwl Segmentation workflow #9

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Common Workflow Language (CWL) Nuclear Segmentation worflow

CWL feature extraction workflow for imaging dataset

## Workflow Steps:

create a [Conda](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#activating-an-environment) environment using python = ">=3.9,<3.12"

#### 1. Install polus-plugins.

- clone a image-tools repository
`git clone https://github.com/camilovelezr/image-tools.git ../`
- cd `image-tools`
- create a new branch
`git checkout -b hd2 remotes/origin/hd2`
- `pip install .`

#### 2. Install workflow-inference-compiler.
- clone a workflow-inference-compiler repository
`git clone https://github.com/camilovelezr/workflow-inference-compiler.git ../`
- cd `workflow-inference-compiler`
- create a new branch
`git checkout -b hd2 remotes/origin/hd2`
- `pip install -e ".[all]"`

#### Note:
Ensure that the [docker-desktop](https://www.docker.com/products/docker-desktop/) is running in the background. To verify that it's operational, you can use the following command:
`docker run -d -p 80:80 docker/getting-started`
This command will launch the `docker/getting-started container` in detached mode (-d flag), exposing port 80 on your local machine (-p 80:80). It's a simple way to test if Docker Desktop is functioning correctly.

## Details
This workflow integrates seven distinct plugins, starting from data retrieval from [Broad Bioimage Benchmark Collection](https://bbbc.broadinstitute.org/), renaming files, correcting uneven illumination, segmenting nuclear objects.

Below are the specifics of the plugins employed in the workflow
1. [bbbc-download-plugin](https://github.com/saketprem/polus-plugins/tree/bbbc_download/utils/bbbc-download-plugin)
2. [file-renaming-tool](https://github.com/PolusAI/image-tools/tree/master/formats/file-renaming-tool)
3. [ome-converter-tool](https://github.com/PolusAI/image-tools/tree/master/formats/ome-converter-tool)
4. [basic-flatfield-estimation-tool](https://github.com/PolusAI/image-tools/tree/master/regression/basic-flatfield-estimation-tool)
5. [apply-flatfield-tool](https://github.com/PolusAI/image-tools/tree/master/transforms/images/apply-flatfield-tool)
6. [kaggle-nuclei-segmentation](https://github.com/hamshkhawar/image-tools/tree/kaggle-nuclei_seg/segmentation/kaggle-nuclei-segmentation)
7. [polus-ftl-label-plugin](https://github.com/hamshkhawar/image-tools/tree/kaggle-nuclei_seg/transforms/images/polus-ftl-label-plugin)

## Execute CWL Segmentation workflow

The parameters for each imaging dataset are pre-defined and stored in JSON format. A Pydantic model in a utils Python file can be utilized to store parameters for any new dataset

`python cwl_workflows/__main__.py --name="BBBC039" --workflow=segmentation`

A directory named `workflow` is generated, encompassing CLTs for each plugin, YAML files, and all outputs are stored within the `outdir` directory.
```
workflows
├── experiment
│ └── cwl_adapters
| experiment.cwl
| experiment.yml
|
└── outdir
└── experiment
├── step 1 BbbcDownload
│ └── outDir
│ └── bbbc.outDir
│ └── BBBC
│ └── BBBC039
│ └── raw
│ ├── Ground_Truth
│ │ ├── masks
│ │ └── metadata
│ └── Images
│ └── images
├── step 2 FileRenaming
│ └── outDir
│ └── rename.outDir
├── step 3 OmeConverter
│ └── outDir
│ └── ome_converter.outDir
├── step 4 BasicFlatfieldEstimation
│ └── outDir
│ └── estimate_flatfield.outDir
├── step 5 ApplyFlatfield
│ └── outDir
│ └── apply_flatfield.outDir
├── step 6 KaggleNucleiSegmentation
│ └── outDir
│ └── kaggle_nuclei_segmentation.outDir
├── step 7 FtlLabel
│ └── outDir
│ └── ftl_plugin.outDir

```
15 changes: 15 additions & 0 deletions bbbc_json/bbbc_segmentation.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"data": {
"BBBC039": {
"name": "BBBC039",
"file_pattern": "/.*/.*/.*/Images/(?P<directory>.*)/.*_{row:c}{col:dd}_s{s:d}_w{channel:d}.*.tif",
"out_file_pattern": "x{row:dd}_y{col:dd}_p{s:dd}_c{channel:d}.tif",
"image_pattern": "images_x{x:dd}_y{y:dd}_p{p:dd}_c{c:d}.ome.tif",
"seg_pattern": "images_x{x:dd}_y{y:dd}_p{p:dd}_c1.ome.tif",
"ff_pattern": "images_x\\(00-15\\)_y\\(01-24\\)_p0\\(1-9\\)_c{c:d}_flatfield.ome.tif",
"df_pattern": "images_x\\(00-15\\)_y\\(01-24\\)_p0\\(1-9\\)_c{c:d}_darkfield.ome.tif",
"group_by": "c",
"map_directory": false
}
}
}
115 changes: 0 additions & 115 deletions cwl_adapters/basic-flatfield-estimation.cwl

This file was deleted.

61 changes: 0 additions & 61 deletions cwl_adapters/bbbcdownload.cwl

This file was deleted.

85 changes: 0 additions & 85 deletions cwl_adapters/file-renaming.cwl

This file was deleted.

Loading
Loading