TODO for bcbio with Debian
==========================

IF YOU ARE READING THIS AND FEEL LIKE HELPING OUT - PLEASE CONTACT US
---------------------------------------------------------------------

The following is a list of packages that are used by at least one of the
workflows bcbio aggregates. A complete coverage by Debian of all these tools
seems increasingly unlikely - their packaging would take longer than the
tools' scientific lifespan, i.e. there is a new generation of problem solving
emerging and/or the workflow as a whole will have mutated in the meantime.

We yet have to find an answer to this challenge.  A series of repositories
of packages that are still waiting in the new queue for their acceptance
in the distribution is available as

 deb http://med.functional.domains/r unstable main contrib non-free
 deb http://med.functional.domains/med unstable main contrib non-free
 deb http://med.functional.domains/nim unstable main contrib non-free
 deb http://med.functional.domains/python unstable main contrib non-free
 deb http://med.functional.domains/science unstable main contrib non-free

Anyway, for quite some of our users this is to just go and use Conda.
We should also consider mixing Conda with Debian, though. But let us
first see where we are with bcbio:

Packages referenced by bioconda
-------------------------------

The following package names are listed on 
 https://bioconda.github.io/recipes/bcbio-nextgen/README.html?highlight=bcbio%20nextgen#recipe-Recipe%20&#x27;bcbio-nextgen&#x27;
For now this is just copy'n'pasted without much further thinking about
it. Some packages like python-dateutil may once have been added for
dependencies that only surfaced as a need because of some better testing
- for python-dateutil (once demanded by seqcluster) I am pretty much
convinced that this is no longer required.

  Unclear
        beautifulsoup4 https://www.crummy.com/software/BeautifulSoup/
        ipyparallel >=6.0.2
        ipython-cluster-helper >=0.6.1
        #py # conda-specific
        python-dateutil >=2.5.0	# likely obsolete?
        pyyaml https://pyyaml.org/wiki/PyYAML
        seqcluster # not yet in new
 To be packaged
	dnapi https://github.com/jnktsj/DNApi
        fadapa https://github.com/fadapa/fadapa
        pycrypto
 Available in Debian but not added as a dependency (yet)
        boto
        htslib	# not called directly?
        #pip # ridiculous
        openssl >=1.1.1c,<1.1.2a
        pytest-mock
        pytest-cov
        seaborn # not called directly?
        yamllint 
 Already a dependency
        pytest # an implicit python build dependency
        arrow
        bioblend
        biopython
        cyvcf2
        geneimpacts
        gffutils
        h5py
        joblib >=0.12
        logbook
        matplotlib
        msgpack-python https://msgpack.org/
        mock
        python
        pandas
        psutil
        pybedtools
        pysam >=0.13.0
        pyvcf
        requests
        scipy >=1.3.0
        statsmodels
        tabulate
        toolz


Packages already accepted
-------------------------

Are no longer shown in this list but are added to the dependencies of
bcbio. See above for conda references to get an impression.

Packages already in the New Queue
---------------------------------

package pizzly (in new queue) (in crash space)
   https://github.com/pmelsted/pizzly
   https://salsa.debian.org/med-team/pizzly
      Was recently rejected from new after a long wait, reuploaded, waiting again

package libjs-scribl (in new queue) (in crash space)
  https://salsa.debian.org/med-team/libjs-scribl
  http://chmille4.github.io/Scribl/
    A surprisingly neat JavaScript library to show genomic features
       Needed for DFSG-compliance of seqcluster

Package MultiQC (in crash space) (rejected from new)
   https://salsa.debian.org/med-team/multiqc
     Addressed https://github.com/ewels/MultiQC/issues/800 by substuting
     with references to CDNs instead of local files.
     The package was placed into contrib because of the dependency of its
     reports on external sources. For that reason and the remote depenency
     on ViennaRNA via seqcluster, also bcbio will be in contrib.
     
Packages just waiting for someone addressing the last mile for the upload
-------------------------------------------------------------------------

package viennarna (in crash space)
    https://github.com/ViennaRNA/ViennaRNA
    https://salsa.debian.org/med-team/vienna-rna/blob/master/debian/changelog
       The package now has a custom license file - redistributable but non-free.
    Needed for testing seqcluster.
    Python and Perl packages are not tested, yet, not needed for seqcluster
dh_auto_install
    Package likely to be uploadable to non-free. Final checks needed.
    Documentation no longer builds with 2.4.14 would be nice to reactivate but not ultimately important for first upload.

package pyomo (in crash space)
  https://github.com/Pyomo/pyomo
  https://salsa.debian.org/science-team/pyomo
    needed by optitype
    Status: Builds if errors in build time tests will be ignored

package python-seqcluster (in crash space)
   https://github.com/lpantano/seqcluster/
   https://salsa.debian.org/med-team/python-seqcluster
      The Python3 transition seems fine now.
      The package needs RNAfold fof the ViennaRNA package for testing. Otherwise it stalls without an error message.
      Dependencies are incomplete / an update wrecked the one immaculate testing.
      When grepping for "conda install":
       ./doc/source/installation.rst:    ~/install/seqcluster/anaconda/conda install seqcluster seqbuster bedtools samtools pip nose numpy scipy pandas pyvcf -c bioconda
       ./Dockerfile:     conda install --yes -c conda-forge -c bioconda scipy seqcluster bedtools samtools pip nose setuptools -q && \
       ./.travis.yml:- conda install --yes -c conda-forge -c bioconda memory_profiler openjdk pysam pybedtools pandas numpy biopython progressbar2 pyyaml bedtools samtools mirtop viennarna -q
      so, for some successful testing it seems like we need viennarna, still. That is not yet in the distribution. If truly dependending on it then bcbio ends up in contrib.

     seqcluster accepted a patch to eliminate a "Free as in beer" graph library from its source tree at ./misc/js/amcharts.js. These are now references to the online location in a CDN.


     Additional requirements:
      python3-pytz ----> packaged but under the name python3-tz

     And there are R packages that should be recommended at least (from https://github.com/lpantano/seqcluster/blob/master/scripts/install_libraries.R)
       # We cover all CRAN packages
       # Most BioC packages are missing - even edgeR ? :
       apparently optional for bcbio: r-bioc-edgeR - missing
       apparently optional for bcbio: r-bioc-HTSFilter - Depends DESeq and edgeR which both in turn depend locfit which makes all non-free
       apparently optional for bcbio: r-bioc-DEGreport - all pre-depends except edgeR are in new
       # novel packages
       apparently optional for bcbio: install.github("hbc/CHBUtils") - Uploaded to new as r-other-chbutils_0.1+git20171026.a226cee-1
       apparently optional for bcbio: r-bioc-isomiRs - (needs r-bioc-DEGreport) all pre-depends except edgeR are in new
       apparently optional for bcbio: install_github('rstudio/rmarkdown')  -- this may be the same we have from CRAN - not checked


Packages presumed easy to package
---------------------------------

package gjh_asl_json
  https://github.com/ghackebeil/gjh_asl_json
    needed by pyomo for testing

package hts_nim_tools
   https://github.com/brentp/hts-nim-tools
   Likely addressed with the other efforts on nim behind mosdepth

package "asl solver"
  http://www.ampl.com/netlib/ampl/solvers.tgz
    needed by gjh_asl_json

r-bioc-edgeR
   https://tracker.debian.org/pkg/r-bioc-edger
	This was removed from buster in February - a bit of a bummer, I tend to think
   https://salsa.debian.org/r-pkg-team/r-bioc-edger
   https://bioconductor.org/packages/release/bioc/html/edgeR.html
     Needed by seqcluster

r-bioc-HTSFilter
   https://bioconductor.org/packages/release/bioc/html/HTSFilter.html
     Needed by seqcluster
   needs r-bioc-edger and r-bioc-deseq (also deseq2) as build dependends which are not in Debian 

r-bioc-DEGreport
   https://bioconductor.org/packages/release/bioc/html/DEGreport.html
     Needed by seqcluster
   needs r-bioc-edger and other dependencies which are not in Debian 


Packages of importance for basic NGS workflows with motivation beyond bcbio
---------------------------------------------------------------------------

package mosdepth
  https://salsa.debian.org/med-team/mosdepth
  several nim libraries are needed. These have all been addressed on https://salsa.debian.org/nim-team
  of which the first already is in the new queue

package oncofuse
  https://salsa.debian.org/med-team/oncofuse
    Stuck over typical Java issue

package optitype
  https://salsa.debian.org/med-team/optitype (rudimentary)
    Stuck over too many dependencies

package vcfanno
  https://salsa.debian.org/med-team/vcfanno
  Not fun because of many GO packages that are still missing,
  "biogo" being one of them.
    Stuck over too many dependencies



Difficult to package to the degree that one is tempted to use conda for the meantime
------------------------------------------------------------------------------------

package snpEff
  http://snpeff.sourceforge.net/
  https://salsa.debian.org/med-team/snpeff/blob/master/debian/changelog
  The packaging needs help with its dependencies:
     apfloat 1.6.3:
       https://github.com/mtommila/apfloat
       http://www.apfloat.org/apfloat_java/maven.html
     charts4j 1.3:
       https://mvnrepository.com/artifact/com.googlecode.charts4j/charts4j/1.3
     com.typesafe.akka:akka-actor 2.0.1
     distlib 0.9.1:
       https://sourceforge.net/projects/statdistlib/?source=navbar

package qsignature
   https://sourceforge.net/p/adamajava/wiki/Home/
   https://sourceforge.net/p/adamajava/wiki/qSignature/
   http://sourceforge.net/projects/adamajava/files/qsignature.tar.bz2/download
   https://sourceforge.net/p/adamajava/code/HEAD/tree/trunk/adamajava/qsignature/

Now at https://github.com/AdamaJava/adamajava

add hisat2_extract_splice_sites.py to the hisat2 package
  -> Done in Git in preparation for 2.1.0-3
  -> Added arch-indep python3-hisat2 package

package sailfish
   https://salsa.debian.org/med-team/sailfish
    FTBFS

package vt
   https://github.com/atks/vt

package Rmath
   https://github.com/atks/Rmath
     needed by vt

package fgbio
  https://github.com/fulcrumgenomics/fgbio
  Anyone much into scala?

package manta
  https://github.com/Illumina/manta/
  https://salsa.debian.org/med-team/manta

package strelka
https://github.com/Illumina/strelka/releases
  - Steffen works on this one
  # Strelka self-compiles htslib with these options - we should cross-check with what we are doing
  ## Addresss sanitizer build options for htslib/samtools
  #set(HTSLIB_CFLAGS '-O0 -g -fsanitize=address -fno-omit-frame-pointer -fno-optimize-sibling-calls')


# An easy one:
package qualimap
  https://salsa.debian.org/med-team/qualimap
  Unfortunately some of the Java code that is needed as predepenceny is simply lost see
     https://alioth-lists.debian.net/pipermail/debian-med-packaging/2016-January/038322.html

References to packages that have not yet chased up
--------------------------------------------------

Likely a "recommends" only:
bcbio/srna/sample.py:    from dnapilib.apred import iterative_adapter_prediction


Packages no longer used by bcbio
--------------------------------

package lumpy
  https://github.com/arq5x/lumpy-sv

