Reading Notes

Reading Notes I: Database Internals - Storage Engines

Fatih Cüre

11 Oct 2024 — 1 min read

Photo by charlesdeluvio / Unsplash

🚸

It is a bullet note rather than a blog post. You probably wouldn't get much or anything.

Database purposes:

Hot data,
Long-lived cold data,
Analytical queries,
Only key-value pairs,
Store time-series data,
Large blob,

Some sources group DBMSs into three major categories,

Online transaction processing (OLTP)
- Handles a large number of user-facing requests and transactions. Queries are often predefined and short-lived,
Online analytical processing (OLTP)
- Handles complex aggregations. Often use for analytics and data warehousing. Complex, long-running, ad-hoc queries,
Hybrid transactional and analytical processing (HTAP)
- Combine OLTP and OLAP

Spatial Locality, One of the Principles of Locality

This basically says that if you fetch data from where it is been, you will likely get its neighbors.

Data Files/ Primary Files can be implemented;

Index-organized tables (IOT)
Heap-organized tables (heap files)
Hash-organized tables (hashed files )

Storage structures common variables;

Buffering
Mutability
Ordering

Read Further

Write Optimizations
- Bitcask
- WiscKey

Hierarchical Navigable Small Worlds: The Meet

As part of a team that is mostly creating solutions for machine learning and LLM-based systems, I love learning the internals after getting some idea for the tools and algorithms, and today's topic is Hierarchical Navigable Small Worlds or HNSW as a short form. 💡 Even if LLM can

Skip List: A Probabilistic Data Structure for Search, Insert and Delete

Skip List is a probabilstic data structure that gives us O(log n) average complexity (and O(n) for worst case for all) for search, insert and delete operations. Why do we need this? If we think about the sorted linked list, when we search some target value we need

TIL: Github does not allow workflows to trigger another workflow

As heading gives the main idea, GitHub does not allow us to use workflow to trigger another workflow but the main reason behind the scene is not completely same. The main reason is, you are using GITHUB_TOKEN for authentication when you don't explicitly define Personal Access Token

Guarding the Chat Flow

Chatbots are very hyped these days, and they come with their own problems. One of them is managing the chat flow. As a party owner or a company, you want to choose the topic that folks are talking about. The participants love natural conversations. The same thing is needed for

Read more

Hierarchical Navigable Small Worlds: The Meet

Skip List: A Probabilistic Data Structure for Search, Insert and Delete

TIL: Github does not allow workflows to trigger another workflow

Guarding the Chat Flow