DyNet-Teaching
Category
Bioinformatics
Program On
Teaching
Version
2.0.3
Author / Distributor
Description
DyNet is a neural network library developed by Carnegie Mellon University and many others. It is written in C++ (with bindings in Python) and is designed to be efficient when run on either CPU or GPU, and to work well with networks that have dynamic structures that change for every training instance. More details at DyNet
Running Program
Also refer to Running Jobs on the teaching cluster
Python 3 version with CPU support only
- Version 2.0.3, installed in /usr/local/apps/gb/dynet/2.0.3-Python-3.6.4-foss-2018a
To use this version of dynet, please first load the module with
ml DyNet/2.0.3-Python-3.6.4-foss-2018a
This module will automatically load Python/3.6.4-foss-2018a and the foss/2018a toolchain.
Example script (myDyNetScript.py) using DyNet for solving the “xor” problem (DyNet Tutorial):
import dynet as dy # define the parameters m = dy.ParameterCollection() W = m.add_parameters((8,2)) V = m.add_parameters((1,8)) b = m.add_parameters((8)) # renew the computation graph dy.renew_cg() # create the network x = dy.vecInput(2) # an input vector of size 2. output = dy.logistic(V*(dy.tanh((W*x)+b))) # define the loss with respect to an output y. y = dy.scalarInput(0) # this will hold the correct answer loss = dy.binary_log_loss(output, y) # create training instances def create_xor_instances(num_rounds=2000): questions = [] answers = [] for round in range(num_rounds): for x1 in 0,1: for x2 in 0,1: answer = 0 if x1==x2 else 1 questions.append((x1,x2)) answers.append(answer) return questions, answers questions, answers = create_xor_instances() # train the network trainer = dy.SimpleSGDTrainer(m) total_loss = 0 seen_instances = 0 for question, answer in zip(questions, answers): x.set(question) y.set(answer) seen_instances += 1 total_loss += loss.value() loss.backward() trainer.update() if (seen_instances > 1 and seen_instances % 100 == 0): print("average loss is:",total_loss / seen_instances)
Sample job submission script (sub.sh) to run this sample script (myDyNetScript.py):
#!/bin/bash
#SBATCH --job-name=dynetjob
#SBATCH --partition=batch
#SBATCH --mail-type=ALL
#SBATCH --mail-user=username@uga.edu
#SBATCH --ntasks=1
#SBATCH --mem=4gb
#SBATCH --time=08:00:00
#SBATCH --output=dynetjob.%j.out
#SBATCH --error=dynetjob.%j.err
cd $SLURM_SUBMIT_DIR
ml DyNet/2.0.3-Python-3.6.4-foss-2018a
python3 myDyNetScript.py
where myDyNetScript.py is the sample script name and it needs to be replaced by the script and options you want to use.
In the real submission script, at least all the above underlined values need to be reviewed or to be replaced by the proper values.
Sample job submission command:
sbatch sub.sh
Documentation
Installation
Version 2.0.3 installed with pip install dynet using Python/3.6.4-foss-2018a on August 14, 2019.
System
64-bit Linux