Merge pull request #131 from graphcore-research/main

Merge updates from main

.flake8

@@ -0,0 +1,6 @@
+[flake8]
+max-line-length = 88
+extend-ignore = E203
+
+copyright-check = True
+copyright-author = Graphcore Ltd

.github/workflows/pre-commit.yaml

@@ -0,0 +1,14 @@
+name: Pre-Commit Checks
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+
+jobs:
+  pre-commit:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - uses: actions/setup-python@v3
+    - uses: pre-commit/[email protected]

.github/workflows/unittest.yaml

@@ -33,4 +33,4 @@ jobs:
           JAX_IPU_MODEL_NUM_TILES: 46
           JAX_PLATFORMS: cpu,ipu
         run: |
-          pytest -s .
+          pytest .

.isort.cfg

@@ -0,0 +1,2 @@
+[settings]
+profile=black

.pre-commit-config.yaml

@@ -0,0 +1,37 @@
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+    -   id: end-of-file-fixer
+        files: (^pyscf_ipu/experimental/)|(^test/)|(\.(cfg|txt|flake8|ini))
+    -   id: trailing-whitespace
+        files: (^pyscf_ipu/experimental/)|(^test/)
+
+-   repo: https://github.com/psf/black
+    rev: 23.9.1
+    hooks:
+    -   id: black-jupyter
+        files: (^pyscf_ipu/experimental/)|(^test/)
+        name: Format code
+
+-   repo: https://github.com/pycqa/isort
+    rev: 5.12.0
+    hooks:
+    -   id: isort
+        files: (^pyscf_ipu/experimental/)|(^test/)
+        name: Sort imports
+
+-   repo: https://github.com/PyCQA/flake8
+    rev: 6.1.0
+    hooks:
+    -   id: flake8
+        files: (^pyscf_ipu/experimental/)|(^test/)
+        name: Check PEP8
+
+-   repo: https://github.com/PyCQA/flake8
+    rev: 6.1.0
+    hooks:
+      - id: flake8
+        args: [--select=C]
+        additional_dependencies: [flake8-copyright]
+        name: copyright check

.pytest.ini

@@ -0,0 +1,2 @@
+[pytest]
+addopts = -s -v --durations=10

CONTRIBUTING.md

@@ -0,0 +1,91 @@
+# Contributing to pyscf-ipu
+
+This project is still evolving but at the moment is focused around a high-performance and easily hackable implementation of Gaussian basis set DFT.
+We hope this is useful for the generation of large-scale datasets needed
+for training machine-learning models. We are interested in hearing any and
+all feedback so feel free to raise any questions, bugs encountered, or enhancement requests as [Issues](https://github.com/graphcore-research/pyscf-ipu/issues).
+
+## Setting up a development environment
+We recommend using the conda package manager as this can automatically enable
+the Graphcore Poplar SDK.  This is particularly useful in VS Code which can automatically
+activate the conda environment in a variety of scenarios:
+* visual debugging
+* running quick experiments in an interactive Jupyter window
+* using VS code for Jupyter notebook development.
+
+The following assumes that you have already set up an install of conda and that
+the conda command is available on your system path.  Refer to your preferred conda
+installer:
+* [miniforge installation](https://github.com/conda-forge/miniforge#install)
+* [conda installation documentation](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html).
+
+1. Create a new conda environment with the same python version as required by the Poplar SDK.
+   For example, on ubuntu 20 use `python=3.8.10`
+   ```bash
+   conda create -n pyscf-ipu python=3.8.10
+   ```
+
+2. Confirm that you have the Poplar SDK installed on your machine and store the location
+   in a temporary shell variable.  The following will test that the SDK is found and
+   configured correctly:
+   ```bash
+   TMP_POPLAR_SDK=/path/to/sdk
+   source $TMP_POPLAR_SDK/enable
+   gc-monitor
+
+3. Activate the environment and make POPLAR_SDK a persistent environment variable.
+   ```bash
+   conda activate pyscf-ipu
+   conda env config vars set POPLAR_SDK=$TMP_POPLAR_SDK
+   ```
+
+4. You have to reactivate the conda environment to use the `$POPLAR_SDK`
+   variable in the environment.
+   ```bash
+   conda deactivate
+   conda activate pyscf-ipu
+   ```
+
+5. Setup the conda environment to automatically enable the Poplar SDK whenever
+   the environment is activated.
+   ```bash
+   mkdir -p $CONDA_PREFIX/etc/conda/activate.d
+   echo "source $POPLAR_SDK/enable" > $CONDA_PREFIX/etc/conda/activate.d/enable.sh
+   ```
+
+6. Check that everything is working by once reactivating the pyscf-ipu
+   environment _in a new shell_ and calling `gc-monitor`:
+   ```bash
+   conda deactivate
+   conda activate pyscf-ipu
+   gc-monitor
+
+7. Install all required packages for developing JAX DFT:
+   ```bash
+   pip install -e ".[ipu,test]"
+   ```
+
+8. Install the pre-commit hooks
+   ```bash
+   pre-commit install
+   ```
+
+9. Create a feature branch, make changes, and when you commit them the
+   pre-commit hooks will run.
+   ```bash
+   git checkout -b feature
+   ...
+   git push --set-upstream origin feature
+   ```
+   The last command will prints a link that you can follow to open a PR.
+
+
+## Testing
+Run all the tests using `pytest`
+```bash
+pytest
+```
+We also use the nbmake package to check our notebooks work in the `IpuModel` environment.  These checks can also be run on IPU hardware equiped machines e.g.:
+```bash
+pytest --nbmake --nbmake-timeout=3000 notebooks/nanoDFT-demo.ipynb
+```

README.md

@@ -3,6 +3,7 @@
 [![notebook-tests](https://github.com/graphcore-research/pyscf-ipu/actions/workflows/notebooks.yaml/badge.svg)](https://github.com/graphcore-research/pyscf-ipu/actions/workflows/notebooks.yaml)
 [![nanoDFT CLI](https://github.com/graphcore-research/pyscf-ipu/actions/workflows/cli.yaml/badge.svg)](https://github.com/graphcore-research/pyscf-ipu/actions/workflows/cli.yaml)
 [![unit tests](https://github.com/graphcore-research/pyscf-ipu/actions/workflows/unittest.yaml/badge.svg)](https://github.com/graphcore-research/pyscf-ipu/actions/workflows/unittest.yaml)
+[![pre-commit checks](https://github.com/graphcore-research/pyscf-ipu/actions/workflows/pre-commit.yaml/badge.svg)](https://github.com/graphcore-research/pyscf-ipu/actions/workflows/pre-commit.yaml)
 
 [**Installation guide**](#installation)
 | [**Example DFT Computations**](#example-dft-computations)
@@ -31,7 +32,7 @@ To generate datasets based on the paper __Repurposing Density Functional Theory
 
 To run the notebook on Graphcore IPU hardware on Paperspace:
 
-[![Run on Gradient](https://assets.paperspace.io/img/gradient-badge.svg)](https://ipu.dev/ipobmC)
+[![Run on Gradient](https://assets.paperspace.io/img/gradient-badge.svg)](https://ipu.dev/YX0jlK)
 
 ### For DFT teaching and learning: nanoDFT
 
@@ -43,6 +44,7 @@ To run the notebook on Graphcore IPU hardware on Paperspace:
 [![Run on Gradient](https://assets.paperspace.io/img/gradient-badge.svg)](https://ipu.dev/ipobmC)
 
 
+
 Additional notebooks in [notebooks](notebooks) demonstrate other aspects of the computation.
 
 ## Installation