Alt-install of Python on Ubuntu

Here is a very good link about how to do it: http://www.rasadacrea.com/en/web-training-courses/howto-install-python

To sum it up:

1. Install the dependencies for python compilation on Ubuntu:

sudo apt-get install build-essential python-dev
sudo apt-get install zlib1g-dev libbz2-dev libcurl4-openssl-dev 
sudo apt-get install libncurses5-dev libsqlite0-dev libreadline-dev 
sudo apt-get install libgdbm-dev libdb4.8-dev libpcap-dev tk-dev 
sudo apt-get -y build-dep python
sudo apt-get -y install libreadline-dev

2. Download and untar the relevant Python version (here 2.7.6):

wget https://www.python.org/ftp/python/2.7.6/Python-2.7.6.tgz
tar xfz Python-2.7.6.tgz

3. cd into the untared Python folder and run the configure and make scripts

cd Python-2.7.6
./configure
make

4. Make alt-install (it is important to make the alt-install and not install so that $python
returns the systsem version (question of stability) )

sudo make altinstall

5. Clean up

cd .. 
sudo rm -r Python-2.7.6*

6. Now you can access to different version of python:

  • the one that came originally:
which python
python
  • and the one you need for your other needs
which python2.7
python2.7

Installing dev versions of python on OS-X

Step1:  Go to the python official download page and download the python interpreter versions you are interested in.

Step2: Install them, by ctrl-clicking on the .mpkg file and choosing to open it with the installer (required to override the fact that the python interpreters are incompatible with the new Guardian secure installation system)

Step3: as described in pip installation guide:

–  issue interpreter version-specific setup tools install:

pythonX.X ez_setup.py

– install version-specific pip installation:

 pythonX.X get-pip.py

Step4: add the pip-X.X specific directory to your path:

nano ~/.bash_profile

and

export PATH=$PATH:/Library/Frameworks/Python.framework/Versions/X.X/bin

Now that you’re done, please verify that the clang is installed and is in your system path. If this is not the case you might experience some trouble installing python modules requiring to be compiled.

Add-on: to install LAPCKs and ATLAS (very useful for Scipy, follow this tutorial )

Installing TitanDB on a personal machine

Just to play around.

Step1: Install HBase:

follow http://hbase.apache.org/book/quickstart.html,

configuration variables:

hbase.rootdir = /opt/hadoop_hb/hbase
hbase.zookeeper.property.dataDir = /opt/hadoop_hb/zookeeper

putting it to the /opt/ file allows other users (such as specific database-attributed users) to access the necessary files without having to mix up with my /usr/ directory files.

Attention: since /opt/ belongs to root don’t forget to

sudo mkdir /opt/hadoop_hb
sudo chown <your_username> /opt/hadoop_hb

if you want to play with hbase from it’s shell

Attention: if youy are using Ubuntu, you will need to modify machine loopback, so that /etc/hosts look like:

127.0.0.1 localhost 
127.0.0.1 your_machine_name

Now you can start the hbase by typing

HBASE_HOME/bin/start-hbase.sh

and check if it is running by typing  in your browser

http://localhost:60010

(unless you’ve changed the default port h base connects itself to)

Step2: Install Elasticsearch:

For this download the elasticsearch.deb package from ElasticSearch official download website and run

sudo dpckg -i elasticsearch.deb

This will install the elasticsearch on your machine and add it to services launched from the start. Now you can check if it is working by typing in your browser (unless you’ve changed the default ports):

http://localhost:9200

Step3: Install TitanDB:

Once the HBase have been installed, download the TitanDB-Hbase .tar.gz and upack it into your directory of choice. once you’ve done with it, you can connect to it via gremling by typing

 gremlin> g = TitanFactory.open('bin/hbase-es.local')

to start it as a part of the embedded rexter server, configure type:

./bin/titan.sh config/titan-server-rexster.xml bin/hbase-es.local

Now you can check that the server is up and running by typing in your browser

http://localhost:8182/graphs

You’re done!

Correct way of modifying the PATH variable in ubuntu

Regardless the fact that many totorials recommend to modify ~./bashrc in order ot perform a permanent modification of PATH for a given user, this is not a way to go. According to the official Ubuntu StackExchange, the way to go is to use the ~/.pam_environment  folder, which is meant specifically for such modifications.

However, pay attention to the fact that you have to follow the pam_environment-specific synthax and thus type

PATH DEFAULT=${PATH}:/path/to/wherever/your/binaries/are

Reproducability in the High-throughtput and computational biology:

Just discovered about the  Potti scandal at Duke (primer for those who have never heard about it before from here: http://en.wikipedia.org/wiki/Anil_Potti)

Currently watching http://videolectures.net/cancerbioinformatics2010_baggerly_irrh/. Some of the extraordinary quotes (approximative though):

If, after a computational analysis, you give a biologist a single gene, unrelated to cancer until now, that correlates the increase of risk of cancer, it is most likely that you would hear something like “No, you’ve got stroma contamination over here: I’ve been studying this gene for years now and I perfectly know that it is completely uncorrelated with cancer”

If, after a computational analysis, you give a biologist a list of hundreds of genes, and you say: here is the genetic signature of cancer, it is most likely that he will just agree with you, because “yeah, this one seems to correlate with that one, so yeah, that makes sense”.

=> This is precisely why I am developping the information flow framework for drug discovery and clinical biology; to make biological sense from the lists of hundreds of perturbed genes.

Forensic Bioinformatics: Here is the raw data, here is the final results. Let’s try to figure out how we get from the raw data to the results, disregarding what they said they did in supdata.

=> Idea: use the chemotherapeutic drug against 60 cell lines pannel to determine specificity  and see if it correlates with the biological knowledge we have about those  cell lines

Let’s use metagenes!!! As matematicians, we know them as PCA, but well, let’s call them metagenes.

Their list and ours: you might see the pattern. Yes, the genes are IDs are off-set by 1.

So, we had a look at the software they were using and it’s documentation. if you want to read the docs, go to my website, because it was me who wrote it, since there were none!

Most of review commitees in biological journals are biologists, they will skip all the part related to the microarray analysis, jump to the results and see if the computational biology results are in agreement with wet lab results.

 

Using LyX for a report

LyX is a very simple and WSYG editor for latex documents, pretty well adapted to the new users, but enclosing the full power of Latex editors (and especially the freedom from all the options distraction that normal WSYG text editors are full of). However it’s first use might require some googling, so here is a couple of tips to speed up the proces:

inserting the references from Mendeley: http://onhavingwords.wordpress.com/2013/03/19/mendeley-lyx/

The margins should be set to 0.98” in order to reproduce the look and feeling of the MS Word / LO Writer.

Installing scikit.sparse on CentOS or Fedora

Step 1: install the METIS library:

1 ) Install cmake as described here:

http://pkgs.org/centos-6-rhel-6/atrpms-testing-x86_64/cmake-2.8.4-1.el6.x86_64.rpm.html,

For the lazy:

– Dowload the latest atrpms-repo rpm from

http://dl.atrpms.net/el6-x86_64/atrpms/stable/

– Install atrpms-repo rpm as an admin:

# sudo rpm -Uvh atrpms-repo*rpm

– Install cmake rpm package:

# yum --enablerepo=atrpms-testing install cmake

2) Install either the GNU make with

# yum install make

or the whole Development tools with

# yum groupinstall "Development Tools"

3) Download METIS from http://glaros.dtc.umn.edu/gkhome/metis/metis/download and follow the instructions in the “install.txt” to actually install it:

– adjust the include.metix.h to adjust the length of ints and floats to better correspond to your architecture and wanted precision (32 or 64 bits)

-execute:

$ make config 
$ make 
# make install

Step 2: Install SuiteSparse:

1) Download the latest version from http://www.cise.ufl.edu/research/sparse/SuiteSparse/, untar it and cd into it

2) Modify the SuiteSparse_config.SuiteSparse_config.mk INSTALL_INCLUDE variable :

INSTALL_INCLUDE = /usr/local/include

3) Build and install it

$ make 
# make install

Step 3: Install the scikit.sparse:

1) Download the latest scikit.sparse from PiPy:

2) in setup.py edit the last statement so that it looks like this:

Extension("scikits.sparse.cholmod",
         ["scikits/sparse/cholmod.pyx"],
         libraries=["cholmod"],
         include_dirs=[np.get_include()].append("/usr/local/include"),
         library_dirs=["/usr/local/lib"],
),

Step 4:

Well, the scikit.sparse imports well at this point, but if we try to import scikits.sparse.cholmod, we have an Import error, where our egg/scikits/sparse/cholmod.so fails to recognize the amd_printf symbol….

Hmmm. Looks like there is still work to be done to get it all working correctly…

Scipy Sparse Matrixes and Linear Algebra

If you need to do a LU decomposition of a Scipy Sparse Matrix (pretty useful for solving systems of differential equations), keep in mind that Cholesky decomposition is generally more stable and rapid for the Hermitian Symmetric positive definite matrixes. In my case, the default LU decompsition method from scipy.sparse.linalg was failing because of the procedural problems.

However you cannot just apply Numpy.linalg.cholesky because the a scipy.sparse.lil_matrix is seen as a linked list and is not a 2D matrix. A solution for this is to use the cholesky decomposition from the scikit.sparse module

Installing and using a neo4j server on a CentOS server

First of all, if you have weird errors while trying to install neo4j server on a machine running under CentOS, it looks like you are trying to perform installation from a folder within your $HOME directory. Which the newly created neo4j user (if you do a default installation) won’t be able to access. In order to avoid it, unpack the neo4j-community-x.x.x package to the /opt/ directory and

You will also need to install the Oracle Java to support the neo4j installation. A good explanation of how to do it can be found on the ubuntu stack overflow. To sum up:

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
sudo apt-get install oracle-java8-set-default

Only install oracle java as default if you don’t have other programs relying on the

If you meet the “java heap out of space error”, what you have to do is to go to the $NEO4J_HOME directorty (where you’ve installed the neo4j files), then to config and then in the neo4j-wrapper.conf file edit the following lines:

wrapper.java.initmemory = 64
wrapper.java.maxmemory = 512 (increase to 1-4 Gb if you have >4Gb of RAM)