plotHeatmap strange scale tick on profiles

[guillaumecharbonnier]

On some profiles produced by plotHeatmap I have noticed that additional samples do not get the same scale ticks as the first one. I am worried that the y-scale is not the same on each sample.

[dpryan79]

Can you make an example matrix available? I'd like to track the cause of this down.

[guillaumecharbonnier]

Here is an example matrix:
example_matrix_for_deepTools_issue_616.txt.gz

This one produces a subtle a subtle mismatch in the tick position with default y-scale:

If I ask for a different y-scale the mismatch is more striking. Also I don't know if it's related but plotHeatmap is not obedient for yMax as I ask for 50 and it uses 60.

[dpryan79]

Thanks, I'll have a look at both issues.

[dpryan79]

As best as I can tell this is a weird matplotlib bug. I get the same problem with the tick marks not lining up, which seems to happen when we override the default ticks that matplotlib produces. On the plus side, this seems to be just the ticks not lining up, the actual data is in the correct place. I don't have the issue where setting --yMax 50 results in 60 being used. I suspect that's another random matplotlib issue.

[guillaumecharbonnier]

I was hoping like you until today:

If that could help I am using deepTools in its own conda environment:
conda create --yes -n deeptools deeptools pysam=0.10.0

I fixed the pysam version according to this issue:
#513

$ conda list -n deeptools
# packages in environment at /gpfs/tagc/home/gcharbonnier/mw/opt/miniconda/envs/deeptools:
#
backports                 1.0                      py35_1    conda-forge
backports.functools_lru_cache 1.4                      py35_1    conda-forge
bcftools                  1.3.1                         1    bioconda
ca-certificates           2017.7.27.1                   0    conda-forge
certifi                   2017.7.27.1              py35_0    conda-forge
curl                      7.54.1                        0    conda-forge
cycler                    0.10.0                   py35_0    conda-forge
dbus                      1.10.22                       0    conda-forge
deeptools                 2.5.3                    py35_0    bioconda
expat                     2.2.1                         0    conda-forge
fontconfig                2.12.1                        4    conda-forge
freetype                  2.7                           1    conda-forge
gettext                   0.19.7                        1    conda-forge
glib                      2.53.5                        0    conda-forge
gst-plugins-base          1.8.0                         0    conda-forge
gstreamer                 1.8.0                         1    conda-forge
htslib                    1.3.2                         0    bioconda
icu                       58.1                          1    conda-forge
jpeg                      9b                            1    conda-forge
krb5                      1.14.2                        0    conda-forge
libffi                    3.2.1                         3    conda-forge
libgcc                    5.2.0                         0  
libgfortran               3.0.0                         1  
libiconv                  1.14                          4    conda-forge
libpng                    1.6.28                        1    conda-forge
libssh2                   1.8.0                         1    conda-forge
libxcb                    1.12                          1    conda-forge
libxml2                   2.9.5                         0    conda-forge
matplotlib                2.1.0                    py35_0    conda-forge
mkl                       2017.0.3                      0  
ncurses                   5.9                          10    conda-forge
numpy                     1.13.1                   py35_0  
openssl                   1.0.2l                        0    conda-forge
pcre                      8.39                          0    conda-forge
pip                       9.0.1                    py35_0    conda-forge
py2bit                    0.2.1                    py35_0    bioconda
pybigwig                  0.2.8                    py35_0    bioconda
pyparsing                 2.2.0                    py35_0    conda-forge
pyqt                      5.6.0                    py35_4    conda-forge
pysam                     0.10.0                   py35_3    bioconda
python                    3.5.4                         0    conda-forge
python-dateutil           2.3                      py35_0    bioconda
pytz                      2017.2                   py35_0    conda-forge
qt                        5.6.2                         3    conda-forge
readline                  6.2                           0    conda-forge
samtools                  1.3.1                         5    bioconda
scipy                     0.19.1              np113py35_0  
setuptools                36.3.0                   py35_0    conda-forge
sip                       4.18                     py35_1    conda-forge
six                       1.11.0                   py35_1    conda-forge
sqlite                    3.13.0                        1    conda-forge
tk                        8.5.19                        2    conda-forge
tornado                   4.5.2                    py35_0    conda-forge
wheel                     0.30.0                     py_1    conda-forge
xorg-libxau               1.0.8                         3    conda-forge
xorg-libxdmcp             1.1.2                         3    conda-forge
xz                        5.2.3                         0    conda-forge
zlib                      1.2.8                         3    conda-forge

[dpryan79]

Can you post or send me the matrix for that last one? It looks like it'll be more useful in debugging this. Thanks for posting the conda env, I might use that for testing.

[guillaumecharbonnier]

Sure. It is too big for github (33MB) so it is available here for 3 weeks:
https://filesender.renater.fr/?s=download&token=dfa301fd-5b21-3c4a-4f09-9992a08ee934
Note the error occurs only on the median profile here. The mean one with default y-scale looks ok.

[dpryan79]

Are you specifying anything other than --yMin/--yMax and --legendLocation? I downloaded the dataset and am using a similar conda environment (you don't need an old pysam version, I had that issue fixed in pysam) but I don't see what you're seeing. For me, S1 is always a tiny bit taller and it has a large spike around the TSS in both samples.

[guillaumecharbonnier]

Yes the first row of profiles is made with the command below, the other two rows use the same command but with --yMin and --yMax. I extracted the profiles to Inkscape to produce the previous png, hence no use of --legendLocation to avoid the flooding legend.

plotHeatmap
--matrixFile out/inp/deepTools/issue_616/issue_616_second_matrix.txt.gz             
--outFileName out/deepTools/plotHeatmap_sortRegions-descend_sortUsing-region_length_averageTypeSummaryPlot-median_colorList-blueCyanYellowOrangeRed_heatmapHeight-15_heatmapWidth-3_whatToShow-phc_xAxisLabel-tss_refPointLabel-0/inp/deepTools/issue_616/issue_616_second_matrix.pdf             
--outFileSortedRegions out/deepTools/plotHeatmap_sortRegions-descend_sortUsing-region_length_averageTypeSummaryPlot-median_colorList-blueCyanYellowOrangeRed_heatmapHeight-15_heatmapWidth-3_whatToShow-phc_xAxisLabel-tss_refPointLabel-0/inp/deepTools/issue_616/issue_616_second_matrix.bed             
--sortRegions descend             
--sortUsing region_length             
--averageTypeSummaryPlot median             
--colorList '#313695,#B6DFEC,#FFF6B1,#F67C4A,#A70226'             
--heatmapHeight 15             
--heatmapWidth 3             
--whatToShow 'plot, heatmap and colorbar'            
--xAxisLabel 'TSS (bp)'             
--refPointLabel '0'

[dpryan79]

@fidelram: The cause of this issue is here. I presume this is a hold-over from allowing different per-plot ylims(). When ticks are modified on one plot they can increase its limits. These altered limits aren't then propagated to the other plots. The simplest solution would be to not decrease the number of ticks. Alternatively, we can store and then reapply the limits on each subgraph. This, however will result in the top tick getting removed sometimes.

Personally, I prefer just leaving the ticks where they are by default.

[fidelram]

@dpryan79 The default behavior of matplotlib some years ago was to put a lot of ticks and thus I tried to reduce them. However, I think that matplotlib is doing now a better job so I agree that the best is to remove this code and let the rendering take care of the ticks. If in the future we want to decrease the number of ticks we should use this https://matplotlib.org/api/ticker_api.html

[dpryan79]

I still need to actually test it, but I've just put the suggested fix in the branch for version 2.5.4.

[dpryan79]

Version 2.5.4 is coming out now.