BitMagic Library samples

Bit-vector operations

./bvsample01 - basic operations to set/get bits with bvector<> container, find cardinality (bit count)
./bvsample02 - set algebra operations, unions, intersections, equivalence of sets, lexicographical comparison
               find_first_mismatch()

./bvsample03 - use of different in-memory bitset compression options, calculate memory footprint
./bvsample04 - serialization of bvector<> to save compressed BLOB to a file or a database
./bvsample05 - use of bvector<>::enumerator - fast iterator to decode bit-vector into indexes of 1 bits
./bvsample06 - allocator example
./bvsample07 - Example how to use logical operations between arrays and bit vectors
./bvsample08 - STL interoperability and set operations with iterators
./bvsample09 - simple binary distance functions and pipeline for complex binary distance formulas
./bvsample10 - extraction of a random subset for Monte Carlo simulations
./bvsample11 - how to compute population counts for ranges in bit-vector (count_range(), count_to(), count_and())
./bvsample12 - review and comparison of different methods to set and clear bits 
./bvsample14 - serialization of bvector<>, logical operations on compressed BLOBs
./bvsample15 - bvector<>::find, search for first and last set bit, dynamic range detection
./bvsample16 - bm::aggregator<> - utility class for fast logical operations on bvector<> groups
               Tech.note: http://bitmagic.io/aggregator.html
./bvsample17 - rank-select operations on bvector<> using rs_index
./bvsample18 - bvector<>::bulk_insert_iterator (efficient bit-vector construction)
./bvsample18a - Import of bvector<> from an external bit-steam ("bmbvimport.h")
./bvsample19 - bvector<>::merge - merge of bit-vectors (can be used for partitioned processing)
./bvsample20 - bvector<>::shift_right() and bvector<>::insert() - bit-shifting / insertion sample
./bvsample21 - bvector<>::shift_left() and bvector<>::erase() - bit-shifting / bit delete sample

./bvsample22 - algorithms for ranges, intervals (bmintervals.h) and bvector<>::find_reverse()
./bvsample23 - bm::interval_enumerator<> - iterator for interval traversal of bit-vector (bmintervals.h) 
./bvsample24 - bm::rank_range_split(..) algorithm to split bit-vector into equal rank(weight) ranges
./bvsample25 - bm::visit_each_bit(..)/bm::visit_each_bit_range(..) bm::for_each_bit(..)/bm::for_each_bit_range(..) 
               algorithms to traverse bit-vector (full or range)
./bvsample26 - immutable bit-vectors, construction, measuring memory savings, bvector<>::freeze()



./bvsetalgebra - Tutorial for Algebra of Sets operations
Tutorial: http://bitmagic.io/set-algebra.html

./bv3vlogic  - Demo for three-valued logic (Kleene) implemented on bit-vectors.


./bvsample01_64 - basic operations with 64-bit bit-vectors, shows how to initialize and compile 
                  BitMagic classes using 48-bit address space.






Sparse vector (bit-transposed vector) for scalar integer types.


./svsample01 - basic example of sparse_vector<> container
./svsample02 - sparse_vector<> serialization example, also shows how to turn on 
               XOR compression (for more details: http://bitmagic.io/bm-xor.html)
               Example shows how to deserialize into read-only (immutable) succinct vector
./svsample03 - sparse_vector<> import, join, extract methods
./svsample04 - sparse_vector<> operations with NULL-able vectors and values
./svsample05 - set transformation algorithm (image) to translate one set to another
               using memory-efficient bit-transposed sparse_vector as a translation function
               Tech.note: http://bitmagic.io/set2set-assoc-remap-opt.html
./svsample06 - load data and search for value(s) in sparse_vector. This sample illustrates back insert 
               iterator, const_iterator and bm::sparse_vector_scanner<> search class 
	       Benchmark different search/scan methods.
	       Tech.note: http://bitmagic.io/sparse-vector-search.html
./svsample07 - insertion sort using bm::sparse_vector_scanner<>::lower_bound()
./svsample08 - sparse_vector<> deserialization to extract specific elements and ranges.
               Example illustrates how to add bookmarks to the serialized BLOB for faster 
               range deserialization
./svsample09 - bm::sparse_vector_find_first_mismatch
./svsample10 - bm::sparse_vector_scanner<>::find_gt() (GT, GE, LT, LE, RANGE) searches

RSC sparse vector (Succinct vector based on Rank-Select methods) for scalar integer types


./rscsample01 - basics of rsc_sparse_vector<> container. Load, unload, serialize.
./rscsample02 - back insert iterator for rsc_sparse_vector<> container. 
                Gather and range deserialization of a rank-select compressed container.
./rscsample03 - how to use rsc_sparse_vector<>::const_iterator 
./rscsample04 - how to construct rsc_sparse_vector<> with a list of known NOT NULL elements and
                randomly set() and inc() them in fast rank-select index mode
./rscsample05 - How to serialize a group of sparse vectors (data frame) using XOR compression
                See also: http://bitmagic.io/bm-xor.html
./rscsample06 - rsc_sparse_vector<>::gather() - random or sorted order extraction
                

String sparse vector (bit-transposed vector) for ASCII(z) strings


./strsvsample01 - basic example of str_sparse_vector<> container, add values, optimize, iterate
./strsvsample02 - insertion sort using str_sparse_vector<> and sparse_vector_scanner<>::lower_bound_str()
./strsvsample02a - How to use fast comparator to sort succinct vector in compressed mode (std::sort)
./strsvsample03 - str_sparse_vector<>::back_insert_iterator is used, then sparse vector
                  remap compressed and saved to disk using serialization API.
./strsvsample04 - Use str_sparse_vector<> container with NULL (unassigned) values
./strsvsample05 - Use str_sparse_vector<> container selective (gather) deserialization and range deserialization.
                  How to add bookmarks to the the serialized BLOB for faster range deserialization.
./strsvsample06 - str_sparse_vector<>::const_iterator in substring mode, 
                  bm::sparse_vector_scanner<> to search for a string and iterate the result-set
./strsvsample07 - bm::sparse_vector_scanner<>::pipeline to bulk search for a thousands of strings,
                  or calculate statistics, benchmarking various search modes and showing how to use 
                  succinct vector remapping to save RAM without loosing performance of search
./strsvsample08 - bm::sparse_vector_scanner<>::bfind_eq_str() binary search with binding
		 bm::str_sparse_vector<>::remap(), bm::str_sparse_vector<>::optimize(), 
                  bm::str_sparse_vector<>::freeze() to reduce heap footprint



More complex examples to illustrate mixed techniques and use cases:


./xsample01 - advanced: different methods of handling of super sparse sets. Benchmarks of set operations for each method. 
               Details at: http://bitmagic.io/case-ER-join.html

./xsample02 - sparse vector based counting sort and histogram construction techniques compared to std::sort()
               Details at: http://bitmagic.io/hist-sort.html

./xsample03 - Search in human genome data (bioinformatics) using bm::sparse_vector<>, bm::rsc_sparse_vector<> 
              Details at: http://bitmagic.io/succinct-snp-search.html 

./xsample04 - Fingerprint pattern matching, SHIFT-AND Bitap algorithm. Search for DNA substrings.
              Details at: http://bitmagic.io/dna-search.html 

./xsample04a - Fast construction of bit-indexes using bulk insert iterator, multi-threaded partitioning and merge.
              Details at: http://bitmagic.io/dna-search-idx.html

./xsample05 - Example how to build a memory compressed dictionary, benchmark how fast BitMagic can search in it.
              As a sample data we use a part of NED catalog of celestial objects.
              Details at: http://bitmagic.io/star-search.html

./xsample06 - Example on how to use DNA 2-bit per bp compression using
              sparse_vector<> and use of bm::sparse_vector_find_first_mismatch()
              to create comparison function. Benchmarks for variants of mismatch search and comparison.
              http://bitmagic.io/dna-compare.html

./xsample07 - DNA k-mer counting example for short (K < 24 ) k-mers.
              Example generates k-mer codes, stores it with 64-bit bvector and computes frequency for each k-mer 
              for term-frequency analysis (TF/IDF). Illustrates use of bvector<>, succinct rsc_sparse_vector<> and
              search scanner for succinct vector. Example shows multi-threading techniques, map-reduce like techniques 
	      when using BitMagic containers.
              

./xsample08 - Example on how to build data model for Genomics Viewer using bit-intervals and succinct
              vectors. Example illustrates a toy rendering with ASCII art and data model range slicing.
              http://bitmagic.io/gen-layout.html

./xsample09 - Another example on histogram construction using memory compact vectors.
              http://bitmagic.io/bm-hist-compr.html

./xsample10 - Example-benchmark on how to implement compressive scrolling of a data set for (Model View Controller).
              Show cases bm::sparse_vector_serializer<>::set_bookmarks( ) and use of range deserialization of sparse vectors.
	      http://bitmagic.io/bm-mvc.html

