Roman Zeyde
253e244a86
Fix cargo fmt for Travis
...
Stable Rust doesn't support `cargo fmt --check` (yet).
2018-07-14 13:02:28 +03:00
Roman Zeyde
2852020c58
Fix leftovers at examples/load.rs
2018-07-13 22:24:14 +03:00
Roman Zeyde
48cd929a99
Remove StoreOptions and enable manual compactions explicitly
2018-07-13 14:23:10 +03:00
Roman Zeyde
b9a4cb4318
Move bulk reader and indexer to separate module
2018-07-12 16:47:39 +03:00
Roman Zeyde
4c7413db9d
Increase open files limit explicitly
...
Otherwise, full compaction may fail.
2018-07-12 14:44:10 +03:00
Roman Zeyde
62a78f18ad
Put "finish" marker after compaction is over
2018-07-12 10:46:33 +03:00
Roman Zeyde
f92dfc808d
Log indexed blk*.dat file path
2018-07-12 09:25:17 +03:00
Roman Zeyde
7b5679db70
Panic if DB exists before running bulk load benchmark
2018-07-11 21:26:12 +03:00
Roman Zeyde
b32e503db2
Rename regular indexing benchmark
2018-07-11 21:26:12 +03:00
Roman Zeyde
dd70a1cc8a
Persist last indexed block marker
2018-07-11 21:26:12 +03:00
Roman Zeyde
10919035bd
Rename bulk load benchmark
2018-07-11 20:44:05 +03:00
Roman Zeyde
4da521b3cd
Refactor bulk load benchmark
...
- bulk writes take ~1h (reading blk*.dat files at ~48MB/s)
- full compaction takes ~45m (writing 37GB of SSTables)
2018-07-11 20:42:57 +03:00
Roman Zeyde
e69686f988
Use multiple bulk index workers
2018-07-10 23:52:26 +03:00
Roman Zeyde
fb372b2915
Split reading from indexing blk*.dat files
...
Also, use a single Vec<Row> for write operation
2018-07-10 15:37:56 +03:00
Roman Zeyde
4c182a0650
Fix formatting
2018-07-09 17:10:24 +03:00
Roman Zeyde
da570a9182
Fix benchmarks
2018-07-08 23:23:36 +03:00
Roman Zeyde
e4a3581da1
Use Path::join() for DB directory
2018-07-06 11:49:25 +03:00
Roman Zeyde
130ced9ae0
Format using nightly rustfmt
2018-07-03 11:46:36 +03:00
Roman Zeyde
82c2caa3af
Fix Daemon usage by benchmarks
2018-07-03 10:52:09 +03:00
Roman Zeyde
7581fd7d1a
Move bulk load helper back to electrs
2018-06-28 23:08:34 +03:00
Roman Zeyde
fd16082925
Don't start bulk loader if already done
2018-06-28 13:41:46 +03:00
Roman Zeyde
70743c9a10
Integrate bulk parser into main server binary
2018-06-28 11:15:30 +03:00
Roman Zeyde
0cceb790b9
Rename parse into bulk
2018-06-28 00:20:24 +03:00
Roman Zeyde
fe03b121f1
Update multiple use formatting
2018-06-27 00:28:50 +03:00
Roman Zeyde
1c70addede
Fix travis
2018-06-26 09:32:35 +03:00
Roman Zeyde
f631bf26af
Dedup FakeStore definition
2018-06-25 23:06:24 +03:00
Roman Zeyde
0de580a300
Perform compaction after blk*.dat parsing is over
2018-06-25 22:16:46 +03:00
Roman Zeyde
8e0a719cee
Replace eprintln!() by error!()
2018-06-25 22:16:22 +03:00
Roman Zeyde
9a5f746cc0
Refactor blk*.dat parser
2018-06-25 14:06:07 +03:00
Roman Zeyde
712c86d405
Fix small typo in bench_index
2018-06-25 10:52:44 +03:00
Roman Zeyde
4200648c8d
Refactor blk*.dat parser
2018-06-25 10:46:30 +03:00
Roman Zeyde
6f3424f734
Move block indexing into parse module
2018-06-24 13:20:09 +03:00
Roman Zeyde
8b0b4323e0
Store indexed rows into a DB
2018-06-24 09:51:18 +03:00
Roman Zeyde
b2b799c4ce
Fix logging during parsing
2018-06-23 23:36:06 +03:00
Roman Zeyde
8785f2747d
Add metrics to parse module
2018-06-23 23:25:32 +03:00
Roman Zeyde
75b3fc969d
Allow closing Channel's internal sender
...
Otherwise, we may get stuck in a deadlock (e.g. parse module):
- the channel's receiver waits for messages
- but the last channel's sender is kept alive (at the channel)
2018-06-23 23:25:32 +03:00
Roman Zeyde
a8e26e6e33
Revert "Update formatting following Rust 1.27 update"
...
This reverts commit 7372f8e239
2018-06-23 19:57:49 +03:00
Roman Zeyde
11da7d884b
Add benchmark for blk*.dat parser
...
Also, create the parser from an existing daemon
2018-06-23 16:17:04 +03:00
Roman Zeyde
7372f8e239
Update formatting following Rust 1.27 update
2018-06-22 18:18:45 +03:00
Roman Zeyde
0afde683c1
Move benchmark tool to examples/
2018-06-14 15:22:47 +03:00
