# go-nemo

**Repository Path**: wanttobeamaster/go-nemo

## Basic Information

- **Project Name**: go-nemo
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2021-04-05
- **Last Updated**: 2022-03-16

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# go-nemo
go wrapper for [nemo](https://github.com/Qihoo360/nemo);

Go-nemo is a storage engine for a consistent distributed kv storege named [elastic cell](https://gitee.com/wanttobeamaster/elasticell).
# nemo
[nemo](https://github.com/Qihoo360/nemo) is a cpp library which encapsulates redis data structure such as k/v, hash, list, set, zset, persistented them on rocksdb. Nemo maps complex data structure to multiple kv entries in rocksdb as plain kv style storage. For instance ,a hash table is represtend as a meta record plus multi data record in rocksdb. Meta record consist of hash table name as rocksdb key, sum of hash table entries as rocksdb value. Data record is encoded with table name + table entry member as rocksdb key, table entry value as rocksdb value. 
# new feature
We add some new feature to nemo for our special use case.

- __storage volume info for complex data struture.__ In meta record, we not only keeps the sum of a hash table or set, but alse records the original storage volume before compresstion.

- __addiatinal rocksdb instances for extra data.__ We add a meta db for range info and a raft db for raft log.

- __data ingest api for online migrate.__ We use rocksdb 'ingest api' to bulk load external data into current nome instance.

# How to build
* Linux with go version higher than 1.8

* Install c++ build tools and c++ dependency libraries.

__CentOS__

```
    yum group install "Development Tools"
    yum install  snappy-devel zlib-devel bzip2-devel gtest-devel jemalloc-devel
```

__Ubuntu__

```
    apt-get install build-essential  
    apt-get install libsnappy-dev zlib1g-dev libbz2-dev libgtest-dev libjemalloc-dev
```

* Install go package

```
    git clone https://gitee.com/wanttobeamaster/c-nemo
    cd c-nemo
    go install .
    git clone --recursive https://gitee.com/wanttobeamaster/go-nemo
    go install .
```
# Caveat
If you want to minimize the object size, use go build flag : -ldflags "-s -w".

The [c-go build tool](https://golang.org/cmd/cgo/) can only compile c or cpp files in the same directory with go file. So you can see many soft-link files in [c-nemo's](https://gitee.com/wanttobeamaster/c-nemo) top level directory.

There are hundreds of cpp files in rocksdb to build together. For now go build tool only runs one thread to complie the cpp files. It will take a few minutes to complete first time build from scratch.