GNWI.CIP

CIP (Computational Intelligence Packages for Mathematica 7 or higher) is an open-source high-level function library for (non-linear) curve fitting and data smoothing (with cubic splines), clustering (k-medoids, ART-2a) and machine learning (multiple linear/polynomial regression, 3-layer feedforward perceptron-type neural networks and support vector machines). In addition it provides several heuristics for the selection of training and test data or methods to estimate the relevance of the data input components. CIP is built on top of the commercial computing platform Mathematica to exploits its algorithmic and graphical capabilities.

The structure of CIP calculations is largely unified: With Get methods data are retrieved or simulated (with the CIP ExperimentalData and CalculatedData package) that are then submitted to a Fit method (of the CIP CurveFit, Cluster, MLR, MPR, SVM or Perceptron package). The result of the latter is a comprehensive info data structure (curveFitInfo, clusterInfo, mlrInfo, mprInfo, svmInfo or perceptronInfo) that can be passed to corresponding Show methods for multiple evaluation purposes like visual inspection of the goodness of fit or to Calculate methods for model related calculations. Similar operations of different packages are denoted in a similar manner to ease their use. Method signatures do mainly contain only structural parameters where technical control parameters may be changed via options if necessary. The straight forward and intuitive scheme Get-Fit-Show/Calculate may easily be remembered. Other CIP packages perform auxiliary tasks like the Graphics package that provides standardized 2D and 3D diagrams.

Since CIP is LGPL open-source it is available in source code (see download section below): Every detail of the implemented methods may be inspected as well as changed or improved.

The CIP design goals were neither maximum speed nor minimum memory consumption but the sketched largely unified and robust access to high-level functions necessary for demonstration purposes. Thus CIP is not an optimized and maximum efficient library for any scientific application although it may and is practically utilized in many operational areas (see comments below). Since CIP is LGPL open-source the library may be used as a starting point for customized and tailored extensions, e.g. an implementation of a multi-core processor support to increase computational speed (many CIP operations are ideally suited for parallel operation).

CI packages

• Utility: The Utility package is a basic package that collects several general methods used by other packages like GetMeanSquaredError which is used by all machine learning related packages. Thus this package is used to decrease redundant code.
• ExperimentalData: The ExperimentalData package provides test data. It makes use of the packages Utility, DataTransformation and CurveFit.
• DataTransformation: CIP performs many internal data transformations for different purposes, e.g. all data that are passed to a machine learning method are scaled before the operation (like ScaleDataMatrix) and re-scaled afterwards (like ScaleDataMatrixReverse). The DataTransformation package comprehends all these methods in a single package. It uses the Utility package.
• Graphics: The Graphics package tailors Mathematica's graphical functions for diagrams and graphical representations. It uses the Utility and DataTransformation packages.
• CalculatedData: The CalculatedData package complements the ExperimentalData package with methods for the generation of simulated data like normally distributed xy-error data around a function for curve fitting with GetXyErrorData. It uses methods from the Utility and DataTransformation packages.
• CurveFit: The CurveFit package tailors Mathematica's built in curve fitting method (NonlinearModelFit) for least-squares minimization and adds a smoothing cubic splines support. Since NonlinearModelFit is an algorithmic state-of-the-art implementation for curve fitting the CurveFit package is well-suited for professional data analysis purposes. It uses the Utility, Graphics, DataTransformation and CalculatedData packages.
• Cluster: The Cluster package tailors Mathematica's built in FindClusters method for clustering purposes and adds an adaptive resonance theory (ART-2a) support. FindClusters is an algorithmic state-of-the-art implementation for k-medoids clustering thus the Cluster package may be used for professional tasks. The package uses the Utility, Graphics and DataTransformation packages.
• MLR/MPR: The MLR/MPR package tailors Mathematica's built in Fit method for multiple linear/polynomial regression (MLR/MPR). This is an algorithmic state-of-the-art implementation so the MLR/MPR package may be used for professional applications. The package uses the Utility, Graphics, DataTransformation and Cluster packages.
• Perceptron: The Perceptron package provides optimization algorithms for three-layer perceptron-type neural networks. It utilizes Mathematica's FindMinimum (ConjugateGradient) or NMinimize (DifferentialEvolution) methods for minimization tasks. The package also provides a backpropagation plus momentum minimization and a classical genetic algorithm based minimization. Although the quality of the minimization algorithms is state-of-the-art the specific calculation setup contains non-optimum redundancies that decrease performance and increase memory consumption. Thus the usage of this package is confined to small data sets with about a thousand I/O pairs for practical application. It uses the Utility, Graphics, DataTransformation and Cluster packages.
• SVM: The SVM package provides constrained optimization algorithms for support vector machines (SVM). It utilizes Mathematica's FindMaximum (InteriorPoint) or NMaximize (DifferentialEvolution) methods for constrained optimization tasks. Although these algorithms are robust they do not exploit any specifics of the support vector objective function to increase optimization convergence speed etc. Therefore a practical application is advised to only very small data sets with less than a thousand I/O pairs. The package uses the Utility, Graphics, DataTransformation and Cluster packages.

Additional information

Achim Zielesny, From Curve Fitting to Machine Learning: An illustrative Guide to scientific Data Analysis and Computational Intelligence, Springer: Intelligent Systems Reference Library, Volume 18, Berlin, 2011.

CIP 1.0 is used for all examples and applications outlined in the book.

Download complete examples and applications: ZIP file with Mathematica/CIP code

Tutorials: New in CIP 1.1/1.2

• Entering Non-Linearity with Multiple Polynomial Regression (MPR): PDF / Mathematica Notebook
• Minimal Model for the WDBC Data: PDF / Mathematica Notebook
• QSPR with MLR: PDF / Mathematica Notebook
• QSPR with small Data Set: PDF / Mathematica Notebook
• Data Cleaning or Splitting: PDF / Mathematica Notebook

User forum

The CIP user forum supports the discussion of problems, comments or ideas.

Citation

Achim Zielesny, CIP - Computational Intelligence Packages, Version 1.0/1.1/1.2, GNWI mbH , Oer-Erkenschwick, Germany, 2011.

Download

CIP 1.0 (for Mathematica 7 or higher): Basic operational release

CIP 1.1 (for Mathematica 7 or higher): Adds MPR and several improvements (see "About.txt")

CIP 1.2 (for Mathematica 7 or higher): Adds minor improvements (see "About.txt")

Dateien