From eefb2b38accffaa625569c9115d5a1f8201bdbec Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 11:25:03 +0100 Subject: [PATCH 01/15] Add cygwin instructions --- wiki/BioSQL.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index b3308b541..279066cf1 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -46,23 +46,31 @@ Installing Required Software You will need to install some database software plus the associated python library so that Biopython can "talk" to the database. In this example we'll talk about the most common choice, MySQL. How you do this -will also depend on your operating system, for example on a Debian or -Ubuntu Linux machine try this: +will also depend on your operating system. +For example on a **Debian or Ubuntu Linux** machine try this: ``` bash sudo apt-get install mysql-common mysql-server python-mysqldb ``` - It will also be important to have perl (to run some of the setup -scripts). Again, on a Debian or Ubuntu Linux machine try this: +scripts). Again, on a **Debian or Ubuntu Linux** machine try this: ``` bash sudo apt-get install perl ``` - You may find perl is already installed. -For Windows users, see [BioSQL on Windows](BioSQL_Windows "wikilink"). +For **Windows** users, see [BioSQL on Windows](BioSQL_Windows "wikilink"). + +For **Cygwin** users, use [apt-cyg](https://github.com/transcode-open/apt-cyg) to install packages: +``` bash +apt-cyg install mysql mysql-server +``` +and to install the driver mysql-connector use [pip]https://pypi.org/project/pip/ + +``` bash +pip install mysql-connector +``` Downloading the BioSQL Schema & Scripts --------------------------------------- From 190dcb74cb4158433351694e7b6c4b9651a2291e Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 11:26:11 +0100 Subject: [PATCH 02/15] forgot parenthesis --- wiki/BioSQL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 279066cf1..5fbaff4ad 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -66,7 +66,7 @@ For **Cygwin** users, use [apt-cyg](https://github.com/transcode-open/apt-cyg) t ``` bash apt-cyg install mysql mysql-server ``` -and to install the driver mysql-connector use [pip]https://pypi.org/project/pip/ +and to install the driver mysql-connector use [pip](https://pypi.org/project/pip/) ``` bash pip install mysql-connector From 65ce8e31b0ed457c31937e9e8c56d1cd87b4dd7c Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 11:29:54 +0100 Subject: [PATCH 03/15] sql not scripts --- wiki/BioSQL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 5fbaff4ad..50f66de92 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -100,7 +100,7 @@ mysqladmin -u root create bioseqdb ``` We can then tell MySQL to load the BioSQL scheme we downloaded above. -Change to the scripts subdirectory from the unzipped BioSQL download, +Change to the sql subdirectory from the unzipped BioSQL download, then: ``` bash From c8e57f6ea0bcac97d55570b92490955f9515e805 Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 11:40:23 +0100 Subject: [PATCH 04/15] Update BioSQL.md --- wiki/BioSQL.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 50f66de92..47d9af47b 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -77,15 +77,19 @@ Downloading the BioSQL Schema & Scripts Once the software is installed, your next task is to setup a database and import the BioSQL schema (i.e. setup the relevant tables within the -database). See [BioSQL downloads](http://www.biosql.org/wiki/Downloads) +database). + +The BioSQL schema can be obtained by: + +* Either [BioSQL downloads](http://www.biosql.org/wiki/Downloads) -- you'll need to unzip the archive. -Alternatively to get the very latest BioSQL, check out their git -repository. Or, navigate to the relevant schema file for your database -and download just that, e.g. -[biosqldb-mysql.sql](https://raw.github.com/biosql/biosql/master/sql/biosqldb-mysql.sql) -for MySQL. You will also want the NCBI Taxonomy loading perl script, -[load\_ncbi\_taxonomy.pl](https://raw.github.com/biosql/biosql/master/scripts/load_ncbi_taxonomy.pl). +* Or to get the **very latest** BioSQL, check out or export their git +repository at (https://github.com/biosql/biosql.git) + +``` bash +svn export https://github.com/biosql/biosql.git/trunk biosql +``` Creating the empty database --------------------------- From 43de72ba204d2bd01fef0c9d6f215889b41373f3 Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 11:53:48 +0100 Subject: [PATCH 05/15] added description of relevant files --- wiki/BioSQL.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 47d9af47b..93a4ae25f 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -90,6 +90,9 @@ repository at (https://github.com/biosql/biosql.git) ``` bash svn export https://github.com/biosql/biosql.git/trunk biosql ``` +The two files that are needed are the following: +1. biosqldb-mysql.sql -- the BioSQL schema -- found inside the **sql** subfolder +2. load_ncbi_taxonomy.pl -- the Perl script to populate the database -- found inside the **scrpts** subfolder Creating the empty database --------------------------- From d7e6acbcc844b1648d01b0462c591328f0b1607a Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 11:55:11 +0100 Subject: [PATCH 06/15] correct small typo "scripts" --- wiki/BioSQL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 93a4ae25f..79dcc6115 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -92,7 +92,7 @@ svn export https://github.com/biosql/biosql.git/trunk biosql ``` The two files that are needed are the following: 1. biosqldb-mysql.sql -- the BioSQL schema -- found inside the **sql** subfolder -2. load_ncbi_taxonomy.pl -- the Perl script to populate the database -- found inside the **scrpts** subfolder +2. load_ncbi_taxonomy.pl -- the Perl script to populate the database -- found inside the **scripts** subfolder Creating the empty database --------------------------- From bde029a7a29548f5b27c56ddde40293b37544925 Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:02:37 +0100 Subject: [PATCH 07/15] added more details about running the perl script --- wiki/BioSQL.md | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 79dcc6115..9a4f300e5 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -77,9 +77,9 @@ Downloading the BioSQL Schema & Scripts Once the software is installed, your next task is to setup a database and import the BioSQL schema (i.e. setup the relevant tables within the -database). +database) as well as the script to populate the database. -The BioSQL schema can be obtained by: +The two files can be obtained by: * Either [BioSQL downloads](http://www.biosql.org/wiki/Downloads) -- you'll need to unzip the archive. @@ -90,9 +90,9 @@ repository at (https://github.com/biosql/biosql.git) ``` bash svn export https://github.com/biosql/biosql.git/trunk biosql ``` -The two files that are needed are the following: -1. biosqldb-mysql.sql -- the BioSQL schema -- found inside the **sql** subfolder -2. load_ncbi_taxonomy.pl -- the Perl script to populate the database -- found inside the **scripts** subfolder +The names of the two files that are needed are the following: +1. biosqldb-mysql.sql -- the BioSQL schema -- found inside the **sql** subdirectory +2. load_ncbi_taxonomy.pl -- the Perl script to populate the database -- found inside the **scripts** subdirectory Creating the empty database --------------------------- @@ -107,8 +107,7 @@ mysqladmin -u root create bioseqdb ``` We can then tell MySQL to load the BioSQL scheme we downloaded above. -Change to the sql subdirectory from the unzipped BioSQL download, -then: +Change to the **sql** subdirectory (see above) and then: ``` bash mysql -u root bioseqdb < biosqldb-mysql.sql @@ -191,11 +190,11 @@ psql biosqldb < biosqldb-pg.sql Run *psql* and type enter *\\d * to see all the entities created. -NCBI Taxonomy -------------- +Populate the database With NCBI Taxonomy +---------------------------------------- -The BioSQL package includes a perl script under -scripts/load\_ncbi\_taxonomy.pl to download and update the taxonomy +The BioSQL package includes a perl script under the +**scripts** subdirectory named **load\_ncbi\_taxonomy.pl** that downloads and updates the taxonomy tables. The script should be able to download the files it needs from the [NCBI taxonomy FTP site](ftp://ftp.ncbi.nih.gov/pub/taxonomy/) automatically. @@ -206,8 +205,7 @@ trying to load sequences into the database. This isn't so important with Biopython 1.49 onwards, where you can instead opt to have the information needed downloaded as needed from Entrez. -To update the NCBI taxonomy, change to the scripts subdirectory from the -unzipped BioSQL download, then: +To update the NCBI taxonomy, change to the **scripts** subdirectory (see above) and then: ``` bash ./load_ncbi_taxonomy.pl --dbname bioseqdb --driver mysql --dbuser root --download true From 8732ab1aaa5833a3a6d73b3281883ac240300a00 Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:03:44 +0100 Subject: [PATCH 08/15] some more package details --- wiki/BioSQL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 9a4f300e5..d6eb6d617 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -62,11 +62,11 @@ You may find perl is already installed. For **Windows** users, see [BioSQL on Windows](BioSQL_Windows "wikilink"). -For **Cygwin** users, use [apt-cyg](https://github.com/transcode-open/apt-cyg) to install packages: +For **Cygwin** users, use [apt-cyg](https://github.com/transcode-open/apt-cyg) to install packages **mysql** and **mysql-server**, ``` bash apt-cyg install mysql mysql-server ``` -and to install the driver mysql-connector use [pip](https://pypi.org/project/pip/) +and to install the driver **mysql-connector** use [pip](https://pypi.org/project/pip/), ``` bash pip install mysql-connector From af4272cafa9d2228f27ea6dbeb5fcf9d37927dc0 Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:05:46 +0100 Subject: [PATCH 09/15] better English --- wiki/BioSQL.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index d6eb6d617..a1286a38f 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -75,9 +75,9 @@ pip install mysql-connector Downloading the BioSQL Schema & Scripts --------------------------------------- -Once the software is installed, your next task is to setup a database -and import the BioSQL schema (i.e. setup the relevant tables within the -database) as well as the script to populate the database. +Once the software is installed, your next task is to setup a database, +import the BioSQL schema (i.e. setup the relevant tables within the +database) as well and finally populate the database. The two files can be obtained by: From 4dcd7ad7d0f86b7d9025cd4655f5ca6873fb4dc3 Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:06:48 +0100 Subject: [PATCH 10/15] remove "as well" typo --- wiki/BioSQL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index a1286a38f..747f36446 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -77,7 +77,7 @@ Downloading the BioSQL Schema & Scripts Once the software is installed, your next task is to setup a database, import the BioSQL schema (i.e. setup the relevant tables within the -database) as well and finally populate the database. +database) and finally populate the database. The two files can be obtained by: From cbd815b875f88d8c607fc9ab75b3db0d663e9e2c Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:09:07 +0100 Subject: [PATCH 11/15] better English --- wiki/BioSQL.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 747f36446..ff0478ca2 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -79,12 +79,12 @@ Once the software is installed, your next task is to setup a database, import the BioSQL schema (i.e. setup the relevant tables within the database) and finally populate the database. -The two files can be obtained by: +In order to do files from the **biosql** project need to be obtained: -* Either [BioSQL downloads](http://www.biosql.org/wiki/Downloads) +* Either from [BioSQL downloads](http://www.biosql.org/wiki/Downloads) -- you'll need to unzip the archive. -* Or to get the **very latest** BioSQL, check out or export their git +* Or to get the **very latest** files, check out (or export) the relevant git repository at (https://github.com/biosql/biosql.git) ``` bash From 82f482830198802e6e019abf81ecc8178c3fe773 Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:10:03 +0100 Subject: [PATCH 12/15] added "so" --- wiki/BioSQL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index ff0478ca2..5aa2c94c5 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -79,7 +79,7 @@ Once the software is installed, your next task is to setup a database, import the BioSQL schema (i.e. setup the relevant tables within the database) and finally populate the database. -In order to do files from the **biosql** project need to be obtained: +In order to do so, files from the **biosql** project need to be obtained: * Either from [BioSQL downloads](http://www.biosql.org/wiki/Downloads) -- you'll need to unzip the archive. From be85e455dc41cfe61f635541a15c045f1b09d6b2 Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:18:48 +0100 Subject: [PATCH 13/15] biosql.ini.sample instructions --- wiki/BioSQL.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 5aa2c94c5..81ede92ef 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -261,19 +261,20 @@ Running the unit tests ---------------------- Because there are so many ways you could have setup your BioSQL -database, you have to tell the unit test a few bits of information by -editing the file Tests/setup\_BioSQL.py and filling in the following +database, you have to tell the unit test a few bits of information. +If you installed biopython using pip then the relevant [**Tests**](https://github.com/biopython/biopython/tree/master/Tests) +folder would not have been copied. If this is the case one can +perform check out (or export) using: + +```bash +svn export https://github.com/biopython/biopython/trunk/Tests +``` +Inside *Tests*, copy the file *biosql.ini.sample* to *biosql.ini* and edit it by filling in the following fields: ``` python DBDRIVER = 'MySQLdb' DBTYPE = 'mysql' -``` - -and a little lower down, - -``` python -DBHOST = 'localhost' DBUSER = 'root' DBPASSWD = '' TESTDB = 'biosql_test' From cb79aa9a86ad13b178e650b945f3e2d521b3406c Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:20:36 +0100 Subject: [PATCH 14/15] inside biosql.ini --- wiki/BioSQL.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 81ede92ef..45bf12427 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -273,11 +273,12 @@ Inside *Tests*, copy the file *biosql.ini.sample* to *biosql.ini* and edit it by fields: ``` python -DBDRIVER = 'MySQLdb' -DBTYPE = 'mysql' -DBUSER = 'root' -DBPASSWD = '' -TESTDB = 'biosql_test' +[mysql] +# Covers DBDRIVER="MySQLdb" and "mysql.connector" etc +dbhost=localhost +dbuser=root +dbpasswd= +testdb=bioseqdb ``` Change these to match your setup. You can then run the BioSQL unit tests From d4f4351596ffeded8a062e303e87195421ddc3fc Mon Sep 17 00:00:00 2001 From: leptoceratops <43407469+leptoceratops@users.noreply.github.com> Date: Thu, 20 Sep 2018 12:23:37 +0100 Subject: [PATCH 15/15] _mysql_connector --- wiki/BioSQL.md | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/wiki/BioSQL.md b/wiki/BioSQL.md index 45bf12427..e66eb780c 100644 --- a/wiki/BioSQL.md +++ b/wiki/BioSQL.md @@ -285,14 +285,25 @@ Change these to match your setup. You can then run the BioSQL unit tests as normal, e.g. ``` bash -python run_tests.py test_BioSQL test_BioSQL_SeqIO +python run_tests.py test_BioSQL_MySQLdb.py test_BioSQL_MySQLdb_online.py ``` +or +``` bash +python run_tests.py test_BioSQL_mysql_connector.py test_BioSQL_mysql_connector_online.py +``` +depending on the driver. -For PostgreSQL, use: +For PostgreSQL in the same file edit the following lines: ``` python -DBDRIVER = 'psycopg2' -DBTYPE = 'pg' +[pg] +# Covers DBDRIVER="psycopg2" etc +# The database and user below are created in .travis.yml +dbhost=localhost +dbuser=biosql_user +dbpasswd=biosql_pass +testdb=biosql_test + ``` Creating a (sub) database