# kann **Repository Path**: step11/kann ## Basic Information - **Project Name**: kann - **Description**: A lightweight C library for artificial neural networks - **Primary Language**: C - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 1 - **Created**: 2023-01-06 - **Last Updated**: 2024-05-29 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README ## Getting Started ```sh # acquire source code and compile git clone https://github.com/attractivechaos/kann cd kann; make # learn unsigned addition (30000 samples; numbers within 10000) seq 30000 | awk -v m=10000 '{a=int(m*rand());b=int(m*rand());print a,b,a+b}' \ | ./examples/rnn-bit -m7 -o add.kan - # apply the model (output 1138429, the sum of the two numbers) echo 400958 737471 | ./examples/rnn-bit -Ai add.kan - ``` ## Introduction KANN is a standalone and lightweight library in C for constructing and training small to medium artificial neural networks such as [multi-layer perceptrons][mlp], [convolutional neural networks][cnn] and [recurrent neural networks][rnn] (including [LSTM][lstm] and [GRU][gru]). It implements graph-based reverse-mode [automatic differentiation][ad] and allows to build topologically complex neural networks with recurrence, shared weights and multiple inputs/outputs/costs. In comparison to mainstream deep learning frameworks such as [TensorFlow][tf], KANN is not as scalable, but it is close in flexibility, has a much smaller code base and only depends on the standard C library. In comparison to other lightweight frameworks such as [tiny-dnn][td], KANN is still smaller, times faster and much more versatile, supporting RNN, VAE and non-standard neural networks that may fail these lightweight frameworks. KANN could be potentially useful when you want to experiment small to medium neural networks in C/C++, to deploy no-so-large models without worrying about [dependency hell][dh], or to learn the internals of deep learning libraries. ### Features * Flexible. Model construction by building a computational graph with operators. Support RNNs, weight sharing and multiple inputs/outputs. * Efficient. Reasonably optimized matrix product and convolution. Support mini-batching and effective multi-threading. Sometimes faster than mainstream frameworks in their CPU-only mode. * Small and portable. As of now, KANN has less than 4000 lines of code in four source code files, with no non-standard dependencies by default. Compatible with ANSI C compilers. ### Limitations * CPU only. As such, KANN is **not** intended for training huge neural networks. * Lack of some common operators and architectures such as batch normalization. * Verbose APIs for training RNNs. ## Installation The KANN library is composed of four files: `kautodiff.{h,c}` and `kann.{h,c}`. You are encouraged to include these files in your source code tree. No installation is needed. To compile examples: ```sh make ``` This generates a few executables in the [examples](examples) directory. ## Documentations Comments in the header files briefly explain the APIs. More documentations can be found in the [doc](doc) directory. Examples using the library are in the [examples](examples) directory. ### A tour of basic KANN APIs Working with neural networks usually involves three steps: model construction, training and prediction. We can use layer APIs to build a simple model: ```c kann_t *ann; kad_node_t *t; t = kann_layer_input(784); // for MNIST t = kad_relu(kann_layer_dense(t, 64)); // a 64-neuron hidden layer with ReLU activation t = kann_layer_cost(t, 10, KANN_C_CEM); // softmax output + multi-class cross-entropy cost ann = kann_new(t, 0); // compile the network and collate variables ``` For this simple feedforward model with one input and one output, we can train it with: ```c int n; // number of training samples float **x; // model input, of size n * 784 float **y; // model output, of size n * 10 // fill in x and y here and then call: kann_train_fnn1(ann, 0.001f, 64, 25, 10, 0.1f, n, x, y); ``` We can save the model to a file with `kann_save()` or use it to classify a MNIST image: ```c float *x; // of size 784 const float *y; // this will point to an array of size 10 // fill in x here and then call: y = kann_apply1(ann, x); ``` Working with complex models requires to use low-level APIs. Please see [01user.md](doc/01user.md) for details. ### A complete example This example learns to count the number of "1" bits in an integer (i.e. popcount): ```c // to compile and run: gcc -O2 this-prog.c kann.c kautodiff.c -lm && ./a.out #include #include #include "kann.h" int main(void) { int i, k, max_bit = 20, n_samples = 30000, mask = (1<