Package TEES :: Package Utils :: Package Libraries :: Module pstat
[hide private]

Module pstat

source code


pstat.py module

#################################################
#######  Written by:  Gary Strangman  ###########
#######  Last modified:  Jun 29, 2001 ###########
#################################################

This module provides some useful list and array manipulation routines
modeled after those found in the |Stat package by Gary Perlman, plus a
number of other useful list/file manipulation functions.  The list-based
functions include:

      abut (source,*args)
      simpleabut (source, addon)
      colex (listoflists,cnums)
      collapse (listoflists,keepcols,collapsecols,fcn1=None,fcn2=None,cfcn=None)
      dm (listoflists,criterion)
      flat (l)
      linexand (listoflists,columnlist,valuelist)
      linexor (listoflists,columnlist,valuelist)
      linedelimited (inlist,delimiter)
      lineincols (inlist,colsize) 
      lineincustcols (inlist,colsizes)
      list2string (inlist)
      makelol(inlist)
      makestr(x)
      printcc (lst,extra=2)
      printincols (listoflists,colsize)
      pl (listoflists)
      printl(listoflists)
      replace (lst,oldval,newval)
      recode (inlist,listmap,cols='all')
      remap (listoflists,criterion)
      roundlist (inlist,num_digits_to_round_floats_to)
      sortby(listoflists,sortcols)
      unique (inlist)
      duplicates(inlist)
      writedelimited (listoflists, delimiter, file, writetype='w')

Some of these functions have alternate versions which are defined only if
Numeric (NumPy) can be imported.  These functions are generally named as
above, with an 'a' prefix.

      aabut (source, *args)
      acolex (a,indices,axis=1)
      acollapse (a,keepcols,collapsecols,sterr=0,ns=0)
      adm (a,criterion)
      alinexand (a,columnlist,valuelist)
      alinexor (a,columnlist,valuelist)
      areplace (a,oldval,newval)
      arecode (a,listmap,col='all')
      arowcompare (row1, row2)
      arowsame (row1, row2)
      asortrows(a,axis=0)
      aunique(inarray)
      aduplicates(inarray)

Currently, the code is all but completely un-optimized.  In many cases, the
array versions of functions amount simply to aliases to built-in array
functions/methods.  Their inclusion here is for function name consistency.


Version: 0.4

Functions [hide private]
 
abut(source, *args)
Like the |Stat abut command.
source code
 
simpleabut(source, addon)
Concatenates two lists as columns and returns the result.
source code
 
colex(listoflists, cnums)
Extracts from listoflists the columns specified in the list 'cnums' (cnums can be an integer, a sequence of integers, or a string-expression that corresponds to a slice operation on the variable x ...
source code
 
collapse(listoflists, keepcols, collapsecols, fcn1=None, fcn2=None, cfcn=None)
Averages data in collapsecol, keeping all unique items in keepcols (using unique, which keeps unique LISTS of column numbers), retaining the unique sets of values in keepcols, the mean for each.
source code
 
dm(listoflists, criterion)
Returns rows from the passed list of lists that meet the criteria in the passed criterion expression (a string as a function of x; e.g., 'x[3]>=9' will return all rows where the 4th column>=9 and "x[2]=='N'" will return rows with column 2 equal to the string 'N').
source code
 
flat(l)
Returns the flattened version of a '2D' list.
source code
 
linexand(listoflists, columnlist, valuelist)
Returns the rows of a list of lists where col (from columnlist) = val (from valuelist) for EVERY pair of values (columnlist[i],valuelists[i]).
source code
 
linexor(listoflists, columnlist, valuelist)
Returns the rows of a list of lists where col (from columnlist) = val (from valuelist) for ANY pair of values (colunmlist[i],valuelist[i[).
source code
 
linedelimited(inlist, delimiter)
Returns a string composed of elements in inlist, with each element separated by 'delimiter.' Used by function writedelimited.
source code
 
lineincols(inlist, colsize)
Returns a string composed of elements in inlist, with each element right-aligned in columns of (fixed) colsize.
source code
 
lineincustcols(inlist, colsizes)
Returns a string composed of elements in inlist, with each element right-aligned in a column of width specified by a sequence colsizes.
source code
 
list2string(inlist, delimit=' ')
Converts a 1D list to a single long string for file output, using the string.join function.
source code
 
makelol(inlist)
Converts a 1D list to a 2D list (i.e., a list-of-lists).
source code
 
makestr(x) source code
 
printcc(lst, extra=2)
Prints a list of lists in columns, customized by the max size of items within the columns (max size of items in col, plus 'extra' number of spaces).
source code
 
printincols(listoflists, colsize)
Prints a list of lists in columns of (fixed) colsize width, where colsize is an integer.
source code
 
pl(listoflists)
Prints a list of lists, 1 list (row) at a time.
source code
 
printl(listoflists)
Alias for pl.
source code
 
replace(inlst, oldval, newval)
Replaces all occurrences of 'oldval' with 'newval', recursively.
source code
 
recode(inlist, listmap, cols=None)
Changes the values in a list to a new set of values (useful when you need to recode data from (e.g.) strings to numbers.
source code
 
remap(listoflists, criterion)
Remaps values in a given column of a 2D list (listoflists).
source code
 
roundlist(inlist, digits)
Goes through each element in a 1D or 2D inlist, and applies the following function to all elements of FloatType ...
source code
 
sortby(listoflists, sortcols)
Sorts a list of lists on the column(s) specified in the sequence sortcols.
source code
 
unique(inlist)
Returns all unique items in the passed list.
source code
 
duplicates(inlist)
Returns duplicate items in the FIRST dimension of the passed list.
source code
 
nonrepeats(inlist)
Returns items that are NOT duplicated in the first dim of the passed list.
source code
 
aabut(source, *args)
Like the |Stat abut command.
source code
 
acolex(a, indices, axis=1)
Extracts specified indices (a list) from passed array, along passed axis (column extraction is default).
source code
 
acollapse(a, keepcols, collapsecols, fcn1=None, fcn2=None, cfcn=None)
Averages data in collapsecol, keeping all unique items in keepcols (using unique, which keeps unique LISTS of column numbers), retaining the unique sets of values in keepcols, the mean for each.
source code
 
adm(a, criterion)
Returns rows from the passed list of lists that meet the criteria in the passed criterion expression (a string as a function of x).
source code
 
isstring(x) source code
 
alinexand(a, columnlist, valuelist)
Returns the rows of an array where col (from columnlist) = val (from valuelist).
source code
 
alinexor(a, columnlist, valuelist)
Returns the rows of an array where col (from columnlist) = val (from valuelist).
source code
 
areplace(a, oldval, newval)
Replaces all occurrences of oldval with newval in array a.
source code
 
arecode(a, listmap, col='all')
Remaps the values in an array to a new set of values (useful when you need to recode data from (e.g.) strings to numbers as most stats packages require.
source code
 
arowcompare(row1, row2)
Compares two rows from an array, regardless of whether it is an array of numbers or of python objects (which requires the cmp function).
source code
 
arowsame(row1, row2)
Compares two rows from an array, regardless of whether it is an array of numbers or of python objects (which requires the cmp function).
source code
 
asortrows(a, axis=0)
Sorts an array "by rows".
source code
 
aunique(inarray)
Returns unique items in the FIRST dimension of the passed array.
source code
 
aduplicates(inarray)
Returns duplicate items in the FIRST dimension of the passed array.
source code
Variables [hide private]
  __package__ = 'TEES.Utils.Libraries'
Function Details [hide private]

abut(source, *args)

source code 

Like the |Stat abut command.  It concatenates two lists side-by-side
and returns the result.  '2D' lists are also accomodated for either argument
(source or addon).  CAUTION:  If one list is shorter, it will be repeated
until it is as long as the longest list.  If this behavior is not desired,
use pstat.simpleabut().

Usage:   abut(source, args)   where args=any # of lists
Returns: a list of lists as long as the LONGEST list past, source on the
         'left', lists in <args> attached consecutively on the 'right'

simpleabut(source, addon)

source code 

Concatenates two lists as columns and returns the result.  '2D' lists
are also accomodated for either argument (source or addon).  This DOES NOT
repeat either list to make the 2 lists of equal length.  Beware of list pairs
with different lengths ... the resulting list will be the length of the
FIRST list passed.

Usage:   simpleabut(source,addon)  where source, addon=list (or list-of-lists)
Returns: a list of lists as long as source, with source on the 'left' and
                 addon on the 'right'

colex(listoflists, cnums)

source code 

Extracts from listoflists the columns specified in the list 'cnums'
(cnums can be an integer, a sequence of integers, or a string-expression that
corresponds to a slice operation on the variable x ... e.g., 'x[3:]' will colex
columns 3 onward from the listoflists).

Usage:   colex (listoflists,cnums)
Returns: a list-of-lists corresponding to the columns from listoflists
         specified by cnums, in the order the column numbers appear in cnums

collapse(listoflists, keepcols, collapsecols, fcn1=None, fcn2=None, cfcn=None)

source code 

Averages data in collapsecol, keeping all unique items in keepcols
(using unique, which keeps unique LISTS of column numbers), retaining the
unique sets of values in keepcols, the mean for each.  Setting fcn1
and/or fcn2 to point to a function rather than None (e.g., stats.sterr, len)
will append those results (e.g., the sterr, N) after each calculated mean.
cfcn is the collapse function to apply (defaults to mean, defined here in the
pstat module to avoid circular imports with stats.py, but harmonicmean or
others could be passed).

Usage:    collapse (listoflists,keepcols,collapsecols,fcn1=None,fcn2=None,cfcn=None)
Returns: a list of lists with all unique permutations of entries appearing in
     columns ("conditions") specified by keepcols, abutted with the result of
     cfcn (if cfcn=None, defaults to the mean) of each column specified by
     collapsecols.

dm(listoflists, criterion)

source code 

Returns rows from the passed list of lists that meet the criteria in the passed criterion expression (a string as a function of x; e.g., 'x[3]>=9' will return all rows where the 4th column>=9 and "x[2]=='N'" will return rows with column 2 equal to the string 'N').

Usage: dm (listoflists, criterion) Returns: rows from listoflists that meet the specified criterion.

flat(l)

source code 

Returns the flattened version of a '2D' list. List-correlate to the a.flat() method of NumPy arrays.

Usage: flat(l)

linexand(listoflists, columnlist, valuelist)

source code 

Returns the rows of a list of lists where col (from columnlist) = val (from valuelist) for EVERY pair of values (columnlist[i],valuelists[i]). len(columnlist) must equal len(valuelist).

Usage: linexand (listoflists,columnlist,valuelist) Returns: the rows of listoflists where columnlist[i]=valuelist[i] for ALL i

linexor(listoflists, columnlist, valuelist)

source code 

Returns the rows of a list of lists where col (from columnlist) = val (from valuelist) for ANY pair of values (colunmlist[i],valuelist[i[). One value is required for each column in columnlist. If only one value exists for columnlist but multiple values appear in valuelist, the valuelist values are all assumed to pertain to the same column.

Usage: linexor (listoflists,columnlist,valuelist) Returns: the rows of listoflists where columnlist[i]=valuelist[i] for ANY i

linedelimited(inlist, delimiter)

source code 

Returns a string composed of elements in inlist, with each element separated by 'delimiter.' Used by function writedelimited. Use ' ' for tab-delimiting.

Usage: linedelimited (inlist,delimiter)

lineincols(inlist, colsize)

source code 

Returns a string composed of elements in inlist, with each element right-aligned in columns of (fixed) colsize.

Usage: lineincols (inlist,colsize) where colsize is an integer

lineincustcols(inlist, colsizes)

source code 

Returns a string composed of elements in inlist, with each element right-aligned in a column of width specified by a sequence colsizes. The length of colsizes must be greater than or equal to the number of columns in inlist.

Usage: lineincustcols (inlist,colsizes) Returns: formatted string created from inlist

list2string(inlist, delimit=' ')

source code 

Converts a 1D list to a single long string for file output, using the string.join function.

Usage: list2string (inlist,delimit=' ') Returns: the string created from inlist

makelol(inlist)

source code 

Converts a 1D list to a 2D list (i.e., a list-of-lists). Useful when you want to use put() to write a 1D list one item per line in the file.

Usage: makelol(inlist) Returns: if l = [1,2,'hi'] then returns [[1],[2],['hi']] etc.

printcc(lst, extra=2)

source code 

Prints a list of lists in columns, customized by the max size of items within the columns (max size of items in col, plus 'extra' number of spaces). Use 'dashes' or '\n' in the list-of-lists to print dashes or blank lines, respectively.

Usage: printcc (lst,extra=2) Returns: None

printincols(listoflists, colsize)

source code 

Prints a list of lists in columns of (fixed) colsize width, where colsize is an integer.

Usage: printincols (listoflists,colsize) Returns: None

pl(listoflists)

source code 

Prints a list of lists, 1 list (row) at a time.

Usage: pl(listoflists) Returns: None

replace(inlst, oldval, newval)

source code 

Replaces all occurrences of 'oldval' with 'newval', recursively.

Usage: replace (inlst,oldval,newval)

recode(inlist, listmap, cols=None)

source code 

Changes the values in a list to a new set of values (useful when you need to recode data from (e.g.) strings to numbers. cols defaults to None (meaning all columns are recoded).

Usage: recode (inlist,listmap,cols=None) cols=recode cols, listmap=2D list Returns: inlist with the appropriate values replaced with new ones

remap(listoflists, criterion)

source code 

Remaps values in a given column of a 2D list (listoflists). This requires a criterion as a function of 'x' so that the result of the following is returned ... map(lambda x: 'criterion',listoflists).

Usage: remap(listoflists,criterion) criterion=string Returns: remapped version of listoflists

roundlist(inlist, digits)

source code 

Goes through each element in a 1D or 2D inlist, and applies the following function to all elements of FloatType ... round(element,digits).

Usage: roundlist(inlist,digits) Returns: list with rounded floats

sortby(listoflists, sortcols)

source code 

Sorts a list of lists on the column(s) specified in the sequence sortcols.

Usage: sortby(listoflists,sortcols) Returns: sorted list, unchanged column ordering

unique(inlist)

source code 

Returns all unique items in the passed list. If the a list-of-lists is passed, unique LISTS are found (i.e., items in the first dimension are compared).

Usage: unique (inlist) Returns: the unique elements (or rows) in inlist

duplicates(inlist)

source code 

Returns duplicate items in the FIRST dimension of the passed list.

Usage: duplicates (inlist)

nonrepeats(inlist)

source code 

Returns items that are NOT duplicated in the first dim of the passed list.

Usage: nonrepeats (inlist)

aabut(source, *args)

source code 

Like the |Stat abut command.  It concatenates two arrays column-wise
and returns the result.  CAUTION:  If one array is shorter, it will be
repeated until it is as long as the other.

Usage:   aabut (source, args)    where args=any # of arrays
Returns: an array as long as the LONGEST array past, source appearing on the
         'left', arrays in <args> attached on the 'right'.

acolex(a, indices, axis=1)

source code 

Extracts specified indices (a list) from passed array, along passed axis (column extraction is default). BEWARE: A 1D array is presumed to be a column-array (and that the whole array will be returned as a column).

Usage: acolex (a,indices,axis=1) Returns: the columns of a specified by indices

acollapse(a, keepcols, collapsecols, fcn1=None, fcn2=None, cfcn=None)

source code 

Averages data in collapsecol, keeping all unique items in keepcols
(using unique, which keeps unique LISTS of column numbers), retaining
the unique sets of values in keepcols, the mean for each.  If stderror or
N of the mean are desired, set either or both parameters to 1.

Usage:   acollapse (a,keepcols,collapsecols,fcn1=None,fcn2=None,cfcn=None)
Returns: unique 'conditions' specified by the contents of columns specified
         by keepcols, abutted with the mean(s) of column(s) specified by
         collapsecols

adm(a, criterion)

source code 

Returns rows from the passed list of lists that meet the criteria in the passed criterion expression (a string as a function of x).

Usage: adm (a,criterion) where criterion is like 'x[2]==37'

alinexand(a, columnlist, valuelist)

source code 

Returns the rows of an array where col (from columnlist) = val (from valuelist). One value is required for each column in columnlist.

Usage: alinexand (a,columnlist,valuelist) Returns: the rows of a where columnlist[i]=valuelist[i] for ALL i

alinexor(a, columnlist, valuelist)

source code 

Returns the rows of an array where col (from columnlist) = val (from valuelist). One value is required for each column in columnlist. The exception is if either columnlist or valuelist has only 1 value, in which case that item will be expanded to match the length of the other list.

Usage: alinexor (a,columnlist,valuelist) Returns: the rows of a where columnlist[i]=valuelist[i] for ANY i

areplace(a, oldval, newval)

source code 

Replaces all occurrences of oldval with newval in array a.

Usage: areplace(a,oldval,newval)

arecode(a, listmap, col='all')

source code 

Remaps the values in an array to a new set of values (useful when you need to recode data from (e.g.) strings to numbers as most stats packages require. Can work on SINGLE columns, or 'all' columns at once.

Usage: arecode (a,listmap,col='all') Returns: a version of array a where listmap[i][0] = (instead) listmap[i][1]

arowcompare(row1, row2)

source code 

Compares two rows from an array, regardless of whether it is an
array of numbers or of python objects (which requires the cmp function).

Usage:   arowcompare(row1,row2)
Returns: an array of equal length containing 1s where the two rows had
         identical elements and 0 otherwise

arowsame(row1, row2)

source code 

Compares two rows from an array, regardless of whether it is an array of numbers or of python objects (which requires the cmp function).

Usage: arowsame(row1,row2) Returns: 1 if the two rows are identical, 0 otherwise.

asortrows(a, axis=0)

source code 

Sorts an array "by rows". This differs from the Numeric.sort() function, which sorts elements WITHIN the given axis. Instead, this function keeps the elements along the given axis intact, but shifts them 'up or down' relative to one another.

Usage: asortrows(a,axis=0) Returns: sorted version of a

aunique(inarray)

source code 

Returns unique items in the FIRST dimension of the passed array. Only works on arrays NOT including string items.

Usage: aunique (inarray)

aduplicates(inarray)

source code 

Returns duplicate items in the FIRST dimension of the passed array. Only works on arrays NOT including string items.

Usage: aunique (inarray)