DyNet-Teaching

Category

Bioinformatics

Program On

Teaching

Version

2.0.3

Author / Distributor

DyNet

Description

DyNet is a neural network library developed by Carnegie Mellon University and many others. It is written in C++ (with bindings in Python) and is designed to be efficient when run on either CPU or GPU, and to work well with networks that have dynamic structures that change for every training instance. More details at DyNet

Running Program

Also refer to Running Jobs on the teaching cluster

Python 3 version with CPU support only

• Version 2.0.3, installed in /usr/local/apps/gb/dynet/2.0.3-Python-3.6.4-foss-2018a

To use this version of dynet, please first load the module with

ml DyNet/2.0.3-Python-3.6.4-foss-2018a

This module will automatically load Python/3.6.4-foss-2018a and the foss/2018a toolchain.

Example script (myDyNetScript.py) using DyNet for solving the “xor” problem (DyNet Tutorial):

import dynet as dy

# define the parameters
m = dy.ParameterCollection()
W = m.add_parameters((8,2))
V = m.add_parameters((1,8))
b = m.add_parameters((8))

# renew the computation graph
dy.renew_cg()

# create the network
x = dy.vecInput(2) # an input vector of size 2.
output = dy.logistic(V*(dy.tanh((W*x)+b)))
# define the loss with respect to an output y.
y = dy.scalarInput(0) # this will hold the correct answer
loss = dy.binary_log_loss(output, y)

# create training instances
def create_xor_instances(num_rounds=2000):
    questions = []
    answers = []
    for round in range(num_rounds):
        for x1 in 0,1:
            for x2 in 0,1:
                answer = 0 if x1==x2 else 1
                questions.append((x1,x2))
                answers.append(answer)
    return questions, answers

questions, answers = create_xor_instances()

# train the network
trainer = dy.SimpleSGDTrainer(m)

total_loss = 0
seen_instances = 0
for question, answer in zip(questions, answers):
    x.set(question)
    y.set(answer)
    seen_instances += 1
    total_loss += loss.value()
    loss.backward()
    trainer.update()
    if (seen_instances > 1 and seen_instances % 100 == 0):
        print("average loss is:",total_loss / seen_instances)

Sample job submission script (sub.sh) to run this sample script (myDyNetScript.py):

#!/bin/bash
#SBATCH --job-name=dynetjob
#SBATCH --partition=batch
#SBATCH --mail-type=ALL
#SBATCH --mail-user=username@uga.edu
#SBATCH --ntasks=1
#SBATCH --mem=4gb
#SBATCH --time=08:00:00
#SBATCH --output=dynetjob.%j.out
#SBATCH --error=dynetjob.%j.err

cd $SLURM_SUBMIT_DIR
ml DyNet/2.0.3-Python-3.6.4-foss-2018a
python3 myDyNetScript.py

where myDyNetScript.py is the sample script name and it needs to be replaced by the script and options you want to use.

In the real submission script, at least all the above underlined values need to be reviewed or to be replaced by the proper values.

Sample job submission command:

sbatch sub.sh

Documentation

DyNet Doc

Installation

Version 2.0.3 installed with pip install dynet using Python/3.6.4-foss-2018a on August 14, 2019.

System

64-bit Linux