


MLPW(1)                  USER COMMANDS                    MLPW(1)



NAME
     mlpw - multi-layer perceptron workbench

SYNOPSIS
     mlpw (-l patterns|-t patterns|-T patterns|-r  patterns)  (-c
     cycles|-m  minutes) (-u updatefreq) (-d) (-a adaptmethod) (-
     e|-s) (-f falrate) (-D) (-A) (-S emphasisfile) mlpmodel

DESCRIPTION
     mlpw provides a general purpose multi-layer perceptron simu-
     lation  designed  to operate within the UCL pattern recogni-
     tion workbench.  The program expects details of the MLP net-
     work  in  a  configuration  file,  the  format  of  which is
     described below.  This file stores the initial configuration
     and  the  current  set  of  weights.   The user supplies the
     basename for the model, and the program uses this to  locate
     the  configuration file as <modelname>.mlp, and the optional
     error file <modelname>.err.  Pattern vectors are supplied in
     two  files  <pattern>.vec which contains the binary floating
     point data, and <pattern>.lab which contains the labels  and
     offsets  into  the  .vec  file for each pattern vector.  The
     format of the patternfiles follows the UCL pattern  recogni-
     tion  workbench  standard.   The  program has three modes of
     operation: learn, test and recognise.  For  each  mode,  the
     user  must  supply  the  basename  of the two pattern vector
     files.

  LEARNING MODE
     In learning mode, the mlp model in the configuration file is
     updated using the patterns in the pattern vector files.  The
     user can choose to run the program for a number of  complete
     cycles  through  the  data, or for a given number of minutes
     (elapsed time).  There are currently two  supported  adapta-
     tion  methods:  method  0 (Standard) has fixed ETA and ALPHA
     values shown to work well for a variety of  previous  tasks;
     methods  1  and  2  (Lai-Wan adaptation, full and part) uses
     changing ETA and ALPHA parameters, chosen as a  function  of
     the direction of adaptation in parameter space.  Methods can
     be mixed on different runs of the program.

  TEST MODE
     In test mode, the mlp model in  the  configuration  file  is
     used to generate outputs for each vector in the pattern vec-
     tor files which is then compared with  the  supplied  label.
     For  the '-t' switch performance statistics are printed, for
     the '-T' switch performance statistics are added to the  PRW
     statistcs  database  '/usr/prw/records'.  In either case the
     statistics are based on the location  of  a  threshold  that
     equalises the miss-rate and the false-alarm rate.  An option
     (-s) produces files  for  plotting  the  Receiver  Operating
     Characteristic.




PRW                     Last change: UCL                        1






MLPW(1)                  USER COMMANDS                    MLPW(1)



  RECOGNITION MODE
     In recognition mode, the mlp model in the configuration file
     is  used  to generate outputs for each vector in the pattern
     vector files.  The input label and the recognised output are
     written  to  the  files  <pattern>.mlp.i and <pattern>.mlp.r
     respectively.

     Options:

     -I         Identify the program name and version.

     -l patterns
                Select learn mode.  Update model by training with
                supplied pattern vector files.

     -t patterns
                Select test mode.  Print  statistics  of  perfor-
                mance  of model on supplied pattern vector files.
                Equal error based on frame percentages.

     -T patterns
                Select test mode.  Save statistics of performance
                of  model on supplied pattern vector files in PRW
                records database. Equal error based on frame per-
                centages.

     -r patterns
                Select recognise mode.  Output labels  into  sup-
                plied pattern vector files.

     -ccycles   (Learn mode only) Train for specified  number  of
                cycles (default 1).

     -mminutes  (Learn mode only) Train for a specified number of
                minutes.

     -uupdatefreq
                (Learn  mode  only)  Perform   adaptation   every
                "updatefreq"  number of pattern vectors (Default:
                1 per cycle).

     -aadaptmethod
                (Learn  mode  only)  Select  adaptation   method.
                "0"=standard,    ETA=0.05,    ALPHA=0.9,   fixed.
                "1"=Lai-Wan method, ETA_ZERO=0.2, LAMDA_ZERO=0.5.
                "2"=Simplified   Lai-Wan   method,  variable  ETA
                (0.01..0.75),   fixed   momentum   ALPHA   (0.9).
                "3"=Quickprop adaptation method by Scott Fahlman.

     -d         Pattern vector labels are discrete  numbers,  not
                continuous  values.   The network must be config-
                ured with as many outputs as there  are  discrete



PRW                     Last change: UCL                        2






MLPW(1)                  USER COMMANDS                    MLPW(1)



                classes.   In discrete mode, the training targets
                are 0.9 on the selected output  and  0.1  on  the
                rest.  In test and recognition modes, the largest
                output is selected as the output class.

     -e         (Learn mode  only)  Append  cycle-by-cycle  error
                values to error file <model>.err.

     -s         (Test  mode  only)   Produce   statistics   files
                <model>.rcx   (False-alarm   rate   as  threshold
                increases), <model>.rcy (Hit  rate  as  threshold
                increases) for plotting ROC.

     -ffalrate  (Test mode only) Produce  performance  statistics
                for  a  given  false-alarm  rate.   Default is to
                choose equal-error point).

     -D         Dump verbose listing of model structure to  stan-
                dard output and exit.

     -A         Use   Array   Processor/Vector   Accelerator   if
                present.   This can actually slow down the train-
                ing of models with small  pattern  vectors.   The
                program  co-operatively shares use of the AP with
                other program using the  vashare(SFS3)  routines.
                Real  work  is performed even when the AP is busy
                with some other process.   The  current  AP  code
                does not work with unlayered nets.

     -Semphasisfile
                (learn mode  only)  Use  a  file  specifying  how
                emphasis  is to be selectively applied to pattern
                vectors that are recognised correctly/incorrectly
                high/low.  The file consists of text lines in the
                following format:
                from <lo> to <hi> target <out> threshold <val> lower <emphlo> higher <emphhi>
                where <lo> to <hi> specifies  a  range  of  label
                values  on  the  pattern vector, and <out> is the
                required target value for the output.  <val> is a
                threshold   for  choosing  one  of  two  emphasis
                values; if  the  current  output  is  lower  than
                <val>,  then  emphasis <emphlo> is applied to the
                weight change, else  if  the  current  output  is
                higher  than  <val>,  then <emphhi> is applied to
                the weight  change.   Thus  the  following  lines
                could  be  used  to emphasize the ON regions of a
                file containing 10% ON pattern vectors:
                from 0.0 to 0.5 target 0.1 threshold 0.2 lower 0 higher 1
                from 0.5 to 1.0 target 0.9 threshold 0.8 lower 10 higher 0

CONFIGURATION FILE




PRW                     Last change: UCL                        3






MLPW(1)                  USER COMMANDS                    MLPW(1)



  STRUCTURE
     The structure of a configuration file is as follows:
             /fIconfiguration/fR
             <layer0>-<layer1>-..-<layerN>
             /fIhistory/fR
             {<cycle> <normalised error> <date>}
             /fIlayer 1/fR
             {<connection> <weight> <delta>}
             /fIlayer 2/fR
             {<connection> <weight> <delta>}
             /fIlayer 3/fR
             {<connection> <weight> <delta>}
     The configuration is  formatted  as  "3-3-1"  meaning  three
     layers,  containing  respectively,  3 units, 3 units and one
     unit.  Each number reflects  the  number  of  units  in  the
     layer.   If the number is simple (e.g. '57') this means that
     all of the units are independent, and may  have  independent
     inputs  from  the  layer below.  If the number is given as a
     product (e.g. '19x3') then this implies  cloning  of  units:
     i.e.  units with identical connections with the layer below,
     and sharing weights and weight changes.  Weights  should  be
     set  up only for one set of cloned units, the others will be
     set-up automatically by mlpw. Product terms in the input  of
     the  input  vector, such that cloned units on the next layer
     will be stepped in time although with shared weights.  Step-
     ping in time occurs when any two adjacent layers have cloned
     elements.

     The history is provided by the mlpw program,  recording  the
     last 10 runs of the program in learning mode.

     The connections are specified as  "2-3  0.3  0.001"  meaning
     unit  2  in  the  layer below is connected to unit 3 in this
     layer with a weight of 0.3  and  a  delta  (last  change  to
     weight)  of  0.001.   Links to thresholds are listed as "t-2
     0.4 0.002" meaning the link from the threshold unit to  unit
     2  in  this  layer has a weight of 0.4 and a delta of 0.002.
     Connections may be made from the direct output of a unit, or
     from the output of the unit in previous cycles: the notation
     is: "N-M" connect unit N at delay 0 to unit M,  "N'-M"  con-
     nect  unit  N at delay 1 to M, "N''-M" connect with delay 2,
     etc.  The current maximum delay is 3.  Delays may be used at
     higher  levels in the network in preference to a wider input
     time window.  Connections may also be 'locked', that is used
     in the network but not updated during training.  Locked con-
     nections are indicated by the use of '=' instead of  '-'  in
     the connection description.

  UNLAYERED NETS
     To set up nets with full  interconnections  unrestricted  by
     layers,  use  the  'unlayered'  keyword in the configuration
     file instead of the 'layer N'  instruction.   For  unlayered



PRW                     Last change: UCL                        4






MLPW(1)                  USER COMMANDS                    MLPW(1)



     nets,  the  configuration  must  contain  three numbers: the
     number of inputs, the number of hidden units (even if  zero)
     and  the  number  of  outputs.  Each hidden unit can be con-
     nected to any input or earlier  hidden  unit.   Each  output
     unit  can  be  connected to any input or hidden unit. Inputs
     can not be connected to other inputs,  nor  outputs  to  any
     other outputs.  The key word 'full' maximises connections as
     in layered nets.  Nodes are  numbered  sequentially  from  1
     (unit  0  is  the threshold unit), so in an unlayered confi-
     guration 3-3-3, the hidden units are 4, 5 & 6 and the output
     units are 7, 8 & 9.

  INITIAL SET-UP
     The minimum required for fully interconnected networks is  a
     specification  of  the  network  configuration and a note of
     full interconnectedness  for  each  specified  layer.   Full
     interconnectedness  can  be specified by simply using "full"
     instead of all of the individual connection lines.   Thus  a
     network  to  produce  a  single output from one frame of 19-
     channel vocoder data can be sepcified simply as:
             configuration
             19-1
             layer 1
             full

VERSION/AUTHOR
     2.9 - Mark Huckvale

SOURCE
     /users/mark/sfs/man/../prw/mlpw.src/mlpw.c

























PRW                     Last change: UCL                        5



