Skip to content

Commit

Permalink
feat: Add non-unique index join iterator (#1908)
Browse files Browse the repository at this point in the history
  • Loading branch information
@joshua-spacetime
joshua-spacetime authored Oct 25, 2024
1 parent fa960b3 commit 839aa99
Show file tree
Hide file tree
Showing 2 changed files with 197 additions and 168 deletions.
2 changes: 1 addition & 1 deletion crates/execution/Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@ license-file = "LICENSE"
description = "The SpacetimeDB query engine"

[dependencies]
spacetimedb-lib.workspace = true
spacetimedb-expr.workspace = true
spacetimedb-lib.workspace = true
spacetimedb-table.workspace = true
Loading

2 comments on commit 839aa99

@github-actions
Copy link

@github-actions github-actions bot commented on 839aa99 Oct 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmarking failed. Please check the workflow run for details.

@github-actions
Copy link

@github-actions github-actions bot commented on 839aa99 Oct 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 6516 6516 0.00% 6550 6550 0.00%
sqlite 5579 5579 0.00% 6033 6033 0.00%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 76682 76682 0.00% 77020 77012 0.01%
stdb_raw u32_u64_str no_index 64 128 2 string 119180 119180 0.00% 119684 119708 -0.02%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 25231 25232 -0.00% 25583 25640 -0.22%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 24199 24199 0.00% 24517 24521 -0.02%
sqlite u32_u64_str no_index 64 128 2 string 144695 144695 0.00% 146157 146157 0.00%
sqlite u32_u64_str no_index 64 128 1 u64 124044 124044 0.00% 125292 125292 0.00%
sqlite u32_u64_str btree_each_column 64 128 1 u64 131361 131361 0.00% 132749 132741 0.01%
sqlite u32_u64_str btree_each_column 64 128 2 string 134494 134494 0.00% 136020 136024 -0.00%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 906633 906430 0.02% 962157 961708 0.05%
stdb_raw u32_u64_str btree_each_column 64 128 1054335 1057834 -0.33% 1087655 1090966 -0.30%
sqlite u32_u64_str unique_0 64 128 398320 398320 0.00% 419248 419244 0.00%
sqlite u32_u64_str btree_each_column 64 128 983637 983637 0.00% 1021811 1021811 0.00%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 153946 153946 0.00% 154012 154012 0.00%
stdb_raw u32_u64_str unique_0 64 16971 16971 0.00% 17021 17025 -0.02%
sqlite u32_u64_str unique_0 1024 1067255 1067255 0.00% 1070519 1070519 0.00%
sqlite u32_u64_str unique_0 64 76219 76201 0.02% 77245 77211 0.04%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47528 47528 0.00% 50214 50214 0.00%
64 bsatn 25509 25509 0.00% 27821 27821 0.00%
16 bsatn 8200 8200 0.00% 9628 9628 0.00%
16 json 12188 12188 0.00% 14126 14126 0.00%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 20838955 21243815 -1.91% 21505705 21939547 -1.98%
stdb_raw u32_u64_str unique_0 64 128 1318226 1318871 -0.05% 1395728 1365317 2.23%
sqlite u32_u64_str unique_0 1024 1024 1802182 1802182 0.00% 1811732 1811732 0.00%
sqlite u32_u64_str unique_0 64 128 128528 128528 0.00% 131408 131408 0.00%
On-disk benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 6521 6521 0.00% 6555 6559 -0.06%
sqlite 5621 5621 0.00% 6123 6123 0.00%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 76687 76687 0.00% 77021 77021 0.00%
stdb_raw u32_u64_str no_index 64 128 2 string 119185 119185 0.00% 119665 119689 -0.02%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 25257 25254 0.01% 25609 25618 -0.04%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 24204 24204 0.00% 24526 24526 0.00%
sqlite u32_u64_str no_index 64 128 1 u64 125965 125965 0.00% 127485 127485 0.00%
sqlite u32_u64_str no_index 64 128 2 string 146616 146616 0.00% 148378 148374 0.00%
sqlite u32_u64_str btree_each_column 64 128 2 string 136616 136616 0.00% 138728 138728 0.00%
sqlite u32_u64_str btree_each_column 64 128 1 u64 133475 133457 0.01% 135337 135315 0.02%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 854969 854497 0.06% 909213 908637 0.06%
stdb_raw u32_u64_str btree_each_column 64 128 1006790 1006975 -0.02% 1069406 1069061 0.03%
sqlite u32_u64_str unique_0 64 128 415857 415857 0.00% 436191 436187 0.00%
sqlite u32_u64_str btree_each_column 64 128 1021898 1021898 0.00% 1058834 1058838 -0.00%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 153951 153951 0.00% 154001 154001 0.00%
stdb_raw u32_u64_str unique_0 64 16976 16976 0.00% 17026 17026 0.00%
sqlite u32_u64_str unique_0 1024 1070323 1070323 0.00% 1074129 1074129 0.00%
sqlite u32_u64_str unique_0 64 77973 77973 0.00% 79259 79259 0.00%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47528 47528 0.00% 50214 50214 0.00%
64 bsatn 25509 25509 0.00% 27821 27821 0.00%
16 bsatn 8200 8200 0.00% 9628 9628 0.00%
16 json 12188 12188 0.00% 14126 14126 0.00%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 19543703 19547973 -0.02% 20264821 20268207 -0.02%
stdb_raw u32_u64_str unique_0 64 128 1271823 1272389 -0.04% 1347875 1348361 -0.04%
sqlite u32_u64_str unique_0 1024 1024 1809743 1809743 0.00% 1818453 1818453 0.00%
sqlite u32_u64_str unique_0 64 128 132654 132654 0.00% 135622 135622 0.00%

Please sign in to comment.