DATABASE SYSTEM IMPLEMENTATION

		     Hector Garcia-Molina
		     Jeffrey D. Ullman
		     Jennifer Widom

		     TABLE OF CONTENTS

1 Introduction to DBMS Implementation 
	1.1 Introducing: The Megatron 2000 Database System 
		1.1.1 Megatron 2000 Implementation Details 
		1.1.2 How Megatron 2000 Executes Queries 
		1.1.3 What's Wrong With Megatron 2000? 
	1.2 Overview of a Database Management System 
		1.2.1 Data-Definition Language Commands 
		1.2.2 Overview of Query Processing 
		1.2.3 Main-Memory Buffers and the Buffer Manager 
		1.2.4 Transaction Processing 
		1.2.5 The Query Processor 
	1.3 Outline of This Book 
		1.3.1 Prerequisites 
		1.3.2 Storage-Management Overview 
		1.3.3 Query-Processing Overview 
		1.3.4 Transaction-Processing Overview 
		1.3.5 Information Integration Overview 
	1.4 Review of Database Models and Languages 
		1.4.1 Relational Model Review 
		1.4.2 SQL Review 
		1.4.3 Relational and Object-Oriented Data 
	1.5 Summary of Chapter 1 
	1.6 References for Chapter 1 

2 Data Storage 
	2.1 The Memory Hierarchy 
		2.1.1 Cache 
		2.1.2 Main Memory 
		2.1.3 Virtual Memory 
		2.1.4 Secondary Storage 
		2.1.5 Tertiary Storage 
		2.1.6 Volatile and Nonvolatile Storage 
		2.1.7 Exercises for Section 2.1 
	2.2 Disks 
		2.2.1 Mechanics of Disks 
		2.2.2 The Disk Controller 
		2.2.3 Disk Storage Characteristics 
		2.2.4 Disk Access Characteristics 
		2.2.5 Writing Blocks 
		2.2.6 Modifying Blocks 
		2.2.7 Exercises for Section 2.2 
	2.3 Using Secondary Storage Effectively 
		2.3.1 The I/O Model of Computation 
		2.3.2 Sorting Data in Secondary Storage 
		2.3.3 Merge-Sort 
		2.3.4 Two-Phase, Multiway Merge-Sort 
		2.3.5 Extension of Multiway Merging to Larger Relations 
		2.3.6 Exercises for Section 2.3 
	2.4 Improving the Access Time of Secondary Storage 
		2.4.1 Organizing Data by Cylinders 
		2.4.2 Using Multiple Disks 
		2.4.3 Mirroring Disks 
		2.4.4 Disk Scheduling and the Elevator Algorithm 
		2.4.5 Prefetching and Large-Scale Buffering 
		2.4.6 Summary of Strategies and Tradeoffs 
		2.4.7 Exercises for Section 2.4 
	2.5 Disk Failures 
		2.5.1 Intermittent Failures 
		2.5.2 Checksums 
		2.5.3 Stable Storage 
		2.5.4 Error-Handling Capabilities of Stable Storage 
		2.5.5 Exercises for Section 2.5 
	2.6 Recovery from Disk Crashes 
		2.6.1 The Failure Model for Disks 
		2.6.2 Mirroring as a Redundancy Technique 
		2.6.3 Parity Blocks 
		2.6.4 An Improvement: RAID 5 
		2.6.5 Coping With Multiple Disk Crashes 
		2.6.6 Exercises for Section 2.6 
	2.7 Summary of Chapter 2 
	2.8 References for Chapter 2 

3 Representing Data Elements 
	3.1 Data Elements and Fields 
		3.1.1 Representing Relational Database Elements 
		3.1.2 Representing Objects 
		3.1.3 Representing Data Elements 
	3.2 Records 
		3.2.1 Building Fixed-Length Records 
		3.2.2 Record Headers 
		3.2.3 Packing Fixed-Length Records into Blocks 
		3.2.4 Exercises for Section 3.2 
	3.3 Representing Block and Record Addresses 
		3.3.1 Client-Server Systems 
		3.3.2 Logical and Structured Addresses 
		3.3.3 Pointer Swizzling 
		3.3.4 Returning Blocks to Disk 
		3.3.5 Pinned Records and Blocks 
		3.3.6 Exercises for Section 3.3 
	3.4 Variable-Length Data and Records 
		3.4.1 Records With Variable-Length Fields 
		3.4.2 Records With Repeating Fields 
		3.4.3 Variable-Format Records 
		3.4.4 Records That Do Not Fit in a Block 
		3.4.5 BLOBS 
		3.4.6 Exercises for Section 3.4 
	3.5 Record Modifications 
		3.5.1 Insertion 
		3.5.2 Deletion 
		3.5.3 Update 
		3.5.4 Exercises for Section 3.5 
	3.6 Summary of Chapter 3 
	3.7 References for Chapter 3 

4 Index Structures 
	4.1 Indexes on Sequential Files 
		4.1.1 Sequential Files 
		4.1.2 Dense Indexes 
		4.1.3 Sparse Indexes 
		4.1.4 Multiple Levels of Index 
		4.1.5 Indexes With Duplicate Search Keys 
		4.1.6 Managing Indexes During Data Modifications 
		4.1.7 Exercises for Section 4.1 
	4.2 Secondary Indexes 
		4.2.1 Design of Secondary Indexes 
		4.2.2 Applications of Secondary Indexes 
		4.2.3 Indirection in Secondary Indexes 
		4.2.4 Document Retrieval and Inverted Indexes 
		4.2.5 Exercises for Section 4.2 
	4.3 B-Trees 
		4.3.1 The Structure of B-trees 
		4.3.2 Applications of B-trees 
		4.3.3 Lookup in B-Trees 
		4.3.4 Range Queries 
		4.3.5 Insertion Into B-Trees 
		4.3.6 Deletion From B-Trees 
		4.3.7 Efficiency of B-Trees 
		4.3.8 Exercises for Section 4.3 
	4.4 Hash Tables 
		4.4.1 Secondary-Storage Hash Tables 
		4.4.2 Insertion Into a Hash Table 
		4.4.3 Hash-Table Deletion 
		4.4.4 Efficiency of Hash Table Indexes 
		4.4.5 Extensible Hash Tables 
		4.4.6 Insertion Into Extensible Hash Tables 
		4.4.7 Linear Hash Tables 
		4.4.8 Insertion Into Linear Hash Tables 
		4.4.9 Exercises for Section 4.4 
	4.5 Summary of Chapter 4 
	4.6 References for Chapter 4 

5 Multidimensional Indexes 
	5.1 Applications Needing Multiple Dimensions 
		5.1.1 Geographic Information Systems 
		5.1.2 Data Cubes 
		5.1.3 Multidimensional Queries in SQL 
		5.1.4 Executing Range Queries Using Conventional Indexes 
		5.1.5 Executing Nearest-Neighbor Queries Using Conventional Indexes 
		5.1.6 Other Limitations of Conventional Indexes 
		5.1.7 Overview of Multidimensional Index Structures 
		5.1.8 Exercises for Section 5.1 
	5.2 Hash-Like Structures for Multidimensional Data 
		5.2.1 Grid Files 
		5.2.2 Lookup in a Grid File 
		5.2.3 Insertion Into Grid Files 
		5.2.4 Performance of Grid Files 
		5.2.5 Partitioned Hash Functions 
		5.2.6 Comparison of Grid Files and Partitioned Hashing 
		5.2.7 Exercises for Section 5.2 
	5.3 Tree-Like Structures for Multidimensional Data 
		5.3.1 Multiple-Key Indexes 
		5.3.2 Performance of Multiple-Key Indexes 
		5.3.3 $kd$-Trees 
		5.3.4 Operations on $kd$-Trees 
		5.3.5 Adapting $kd$-Trees to Secondary Storage 
		5.3.6 Quad Trees 
		5.3.7 R-Trees 
		5.3.8 Operations on R-trees 
		5.3.9 Exercises for Section 5.3 
	5.4 Bitmap Indexes 
		5.4.1 Motivation for Bitmap Indexes 
		5.4.2 Compressed Bitmaps 
		5.4.3 Operating on Run-Length-Encoded Bit-Vectors 
		5.4.4 Managing Bitmap Indexes 
		5.4.5 Exercises for Section 5.4 
	5.5 Summary of Chapter 5 
	5.6 References for Chapter 5 

6 Query Execution 
	6.1 An Algebra for Queries 
		6.1.1 Union, Intersection, and Difference 
		6.1.2 The Selection Operator 
		6.1.3 The Projection Operator 
		6.1.4 The Product of Relations 
		6.1.5 Joins 
		6.1.6 Duplicate Elimination 
		6.1.7 Grouping and Aggregation 
		6.1.8 The Sorting Operator 
		6.1.9 Expression Trees 
		6.1.10 Exercises for Section 6.1 
	6.2 Introduction to Physical-Query-Plan Operators 
		6.2.1 Scanning Tables 
		6.2.2 Sorting While Scanning Tables 
		6.2.3 The Model of Computation for Physical Operators 
		6.2.4 Parameters for Measuring Costs 
		6.2.5 I/O Cost for Scan Operators 
		6.2.6 Iterators for Implementation of Physical Operators 
	6.3 One-Pass Algorithms for Database Operations 
		6.3.1 One-Pass Algorithms for Tuple-at-a-Time Operations 
		6.3.2 One-Pass Algorithms for Unary, Full-Relation Operations 
		6.3.3 One-Pass Algorithms for Binary Operations 
		6.3.4 Exercises for Section 6.3 
	6.4 Nested-Loop Joins 
		6.4.1 Tuple-Based Nested-Loop Join 
		6.4.2 An Iterator for Tuple-Based Nested-Loop Join 
		6.4.3 A Block-Based Nested-Loop Join Algorithm 
		6.4.4 Analysis of Nested-Loop Join 
		6.4.5 Summary of Algorithms so Far 
		6.4.6 Exercises for Section 6.4 
	6.5 Two-Pass Algorithms Based on Sorting 
		6.5.1 Duplicate Elimination Using Sorting 
		6.5.2 Grouping and Aggregation Using Sorting 
		6.5.3 A Sort-Based Union Algorithm 
		6.5.4 Sort-Based Algorithms for Intersection and Difference 
		6.5.5 A Simple Sort-Based Join Algorithm 
		6.5.6 Analysis of Simple Sort-Join 
		6.5.7 A More Efficient Sort-Based Join 
		6.5.8 Summary of Sort-Based Algorithms 
		6.5.9 Exercises for Section 6.5 
	6.6 Two-Pass Algorithms Based on Hashing 
		6.6.1 Partitioning Relations by Hashing 
		6.6.2 A Hash-Based Algorithm for Duplicate Elimination 
		6.6.3 A Hash-Based Algorithm for Grouping and Aggregation 
		6.6.4 Hash-Based Algorithms for Union, Intersection, and Difference 
		6.6.5 The Hash-Join Algorithm 
		6.6.6 Saving Some Disk I/O's 
		6.6.7 Summary of Hash-Based Algorithms 
		6.6.8 Exercises for Section 6.6 
	6.7 Index-Based Algorithms 
		6.7.1 Clustering and Nonclustering Indexes 
		6.7.2 Index-Based Selection 
		6.7.3 Joining by Using an Index 
		6.7.4 Joins Using a Sorted Index 
		6.7.5 Exercises for Section 6.7 
	6.8 Buffer Management 
		6.8.1 Buffer Management Architecture 
		6.8.2 Buffer Management Strategies 
		6.8.3 The Relationship Between Physical Operator Selection and Buffer Management 
		6.8.4 Exercises for Section 6.8 
	6.9 Algorithms Using More Than Two Passes 
		6.9.1 Multipass Sort-Based Algorithms 
		6.9.2 Performance of Multipass, Sort-Based Algorithms 
		6.9.3 Multipass Hash-Based Algorithms 
		6.9.4 Performance of Multipass Hash-Based Algorithms 
		6.9.5 Exercises for Section 6.9 
	6.10 Parallel Algorithms for Relational Operations 
		6.10.1 Models of Parallelism 
		6.10.2 Tuple-at-a-Time Operations in Parallel 
		6.10.3 Parallel Algorithms for Full-Relation Operations 
		6.10.4 Performance of Parallel Algorithms 
		6.10.5 Exercises for Section 6.10 
	6.11 Summary of Chapter 6 
	6.12 References for Chapter 6 

7 The Query Compiler 
	7.1 Parsing 
		7.1.1 Syntax Analysis and Parse Trees 
		7.1.2 A Grammar for a Simple Subset of SQL 
		7.1.3 The Preprocessor 
		7.1.4 Exercises for Section 7.1 
	7.2 Algebraic Laws for Improving Query Plans 
		7.2.1 Commutative and Associative Laws 
		7.2.2 Laws Involving Selection 
		7.2.3 Pushing Selections 
		7.2.4 Laws Involving Projection 
		7.2.5 Laws About Joins and Products 
		7.2.6 Laws Involving Duplicate Elimination 
		7.2.7 Laws Involving Grouping and Aggregation 
		7.2.8 Exercises for Section 7.2 
	7.3 From Parse Trees to Logical Query Plans 
		7.3.1 Conversion to Relational Algebra 
		7.3.2 Removing Subqueries From Conditions 
		7.3.3 Improving the Logical Query Plan 
		7.3.4 Grouping Associative/Commutative Operators 
		7.3.5 Exercises for Section 7.3 
	7.4 Estimating the Cost of Operations 
		7.4.1 Estimating Sizes of Intermediate Relations 
		7.4.2 Estimating the Size of a Projection 
		7.4.3 Estimating the Size of a Selection 
		7.4.4 Estimating the Size of a Join 
		7.4.5 Natural Joins With Multiple Join Attributes 
		7.4.6 Joins of Many Relations 
		7.4.7 Estimating Sizes for Other Operations 
		7.4.8 Exercises for Section 7.4 
	7.5 Introduction to Cost-Based Plan Selection 
		7.5.1 Obtaining Estimates for Size Parameters 
		7.5.2 Incremental Computation of Statistics 
		7.5.3 Heuristics for Reducing the Cost of Logical Query Plans 
		7.5.4 Approaches to Enumerating Physical Plans 
		7.5.5 Exercises for Section 7.5 
	7.6 Choosing an Order for Joins 
		7.6.1 Significance of Left and Right Join Arguments 
		7.6.2 Join Trees 
		7.6.3 Left-Deep Join Trees 
		7.6.4 Dynamic Programming to Select a Join Order and Grouping 
		7.6.5 Dynamic Programming With More Detailed Cost Functions 
		7.6.6 A Greedy Algorithm for Selecting a Join Order 
		7.6.7 Exercises for Section 7.6 
	7.7 Completing the Physical-Query-Plan Selection 
		7.7.1 Choosing a Selection Method 
		7.7.2 Choosing a Join Method 
		7.7.3 Pipelining Versus Materialization 
		7.7.4 Pipelining Unary Operations 
		7.7.5 Pipelining Binary Operations 
		7.7.6 Notation for Physical Query Plans 
		7.7.7 Ordering of Physical Operations 
		7.7.8 Exercises for Section 7.7 
	7.8 Summary of Chapter 7 
	7.9 References for Chapter 7 

8 Coping With System Failures 
	8.1 Issues and Models for Resilient Operation 
		8.1.1 Failure Modes 
		8.1.2 More About Transactions 
		8.1.3 Correct Execution of Transactions 
		8.1.4 The Primitive Operations of Transactions 
		8.1.5 Exercises for Section 8.1 
	8.2 Undo Logging 
		8.2.1 Log Records 
		8.2.2 The Undo-Logging Rules 
		8.2.3 Recovery Using Undo Logging 
		8.2.4 Checkpointing 
		8.2.5 Nonquiescent Checkpointing 
		8.2.6 Exercises for Section 8.2 
	8.3 Redo Logging 
		8.3.1 The Redo-Logging Rule 
		8.3.2 Recovery With Redo Logging 
		8.3.3 Checkpointing a Redo Log 
		8.3.4 Recovery With a Checkpointed Redo Log 
		8.3.5 Exercises for Section 8.3 
	8.4 Undo/Redo Logging 
		8.4.1 The Undo/Redo Rules 
		8.4.2 Recovery With Undo/Redo Logging 
		8.4.3 Checkpointing an Undo/Redo Log 
		8.4.4 Exercises for Section 8.4 
	8.5 Protecting Against Media Failures 
		8.5.1 The Archive 
		8.5.2 Nonquiescent Archiving 
		8.5.3 Recovery Using an Archive and Log 
		8.5.4 Exercises for Section 8.5 
	8.6 Summary of Chapter 8 
	8.7 References for Chapter 8 

9 Concurrency Control 
	9.1 Serial and Serializable Schedules 
		9.1.1 Schedules 
		9.1.2 Serial Schedules 
		9.1.3 Serializable Schedules 
		9.1.4 The Effect of Transaction Semantics 
		9.1.5 A Notation for Transactions and Schedules 
		9.1.6 Exercises for Section 9.1 
	9.2 Conflict-Serializability 
		9.2.1 Conflicts 
		9.2.2 Precedence Graphs and a Test for Conflict-Serializability 
		9.2.3 Why the Precedence-Graph Test Works 
		9.2.4 Exercises for Section 9.2 
	9.3 Enforcing Serializability by Locks 
		9.3.1 Locks 
		9.3.2 The Locking Scheduler 
		9.3.3 Two-Phase Locking 
		9.3.4 Why Two-Phase Locking Works 
		9.3.5 Exercises for Section 9.3 
	9.4 Locking Systems With Several Lock Modes 
		9.4.1 Shared and Exclusive Locks 
		9.4.2 Compatibility Matrices 
		9.4.3 Upgrading Locks 
		9.4.4 Update Locks 
		9.4.5 Increment Locks 
		9.4.6 Exercises for Section 9.4 
	9.5 An Architecture for a Locking Scheduler 
		9.5.1 A Scheduler That Inserts Lock Actions 
		9.5.2 The Lock Table 
		9.5.3 Exercises for Section 9.5 
	9.6 Managing Hierarchies of Database Elements 
		9.6.1 Locks With Multiple Granularity 
		9.6.2 Warning Locks 
		9.6.3 Phantoms and Handling Insertions Correctly 
		9.6.4 Exercises for Section 9.6 
	9.7 The Tree Protocol 
		9.7.1 Motivation for Tree-Based Locking 
		9.7.2 Rules for Access to Tree-Structured Data 
		9.7.3 Why the Tree Protocol Works 
		9.7.4 Exercises for Section 9.7 
	9.8 Concurrency Control by Timestamps 
		9.8.1 Timestamps 
		9.8.2 Physically Unrealizable Behaviors 
		9.8.3 Problems With Dirty Data 
		9.8.4 The Rules for Timestamp-Based Scheduling 
		9.8.5 Multiversion Timestamps 
		9.8.6 Timestamps and Locking 
		9.8.7 Exercises for Section 9.8 
	9.9 Concurrency Control by Validation 
		9.9.1 Architecture of a Validation-Based Scheduler 
		9.9.2 The Validation Rules 
		9.9.3 Comparison of Three Concurrency-Control Mechanisms 
		9.9.4 Exercises for Section 9.9 
	9.10 Summary of Chapter 9 
	9.11 References for Chapter 9 

10 More About Transaction Management 
	10.1 Transactions that Read Uncommitted Data 
		10.1.1 The Dirty-Data Problem 
		10.1.2 Cascading Rollback 
		10.1.3 Managing Rollbacks 
		10.1.4 Group Commit 
		10.1.5 Logical Logging 
		10.1.6 Exercises for Section 10.1 
	10.2 View Serializability 
		10.2.1 View Equivalence 
		10.2.2 Polygraphs and the Test for View-Serializability 
		10.2.3 Testing for View-Serializability 
		10.2.4 Exercises for Section 10.2 
	10.3 Resolving Deadlocks 
		10.3.1 Deadlock Detection by Timeout 
		10.3.2 The Waits-For Graph 
		10.3.3 Deadlock Prevention by Ordering Elements 
		10.3.4 Detecting Deadlocks by Timestamps 
		10.3.5 Comparison of Deadlock-Management Methods 
		10.3.6 Exercises for Section 10.3 
	10.4 Distributed Databases 
		10.4.1 Distribution of Data 
		10.4.2 Distributed Transactions 
		10.4.3 Data Replication 
		10.4.4 Distributed Query Optimization 
		10.4.5 Exercises for Section 10.4 
	10.5 Distributed Commit 
		10.5.1 Supporting Distributed Atomicity 
		10.5.2 Two-Phase Commit 
		10.5.3 Recovery of Distributed Transactions 
		10.5.4 Exercises for Section 10.5 
	10.6 Distributed Locking 
		10.6.1 Centralized Lock Systems 
		10.6.2 A Cost Model for Distributed Locking Algorithms 
		10.6.3 Locking Replicated Elements 
		10.6.4 Primary-Copy Locking 
		10.6.5 Global Locks From Local Locks 
		10.6.6 Exercises for Section 10.6 
	10.7 Long-Duration Transactions 
		10.7.1 Problems of Long Transactions 
		10.7.2 Sagas 
		10.7.3 Compensating Transactions 
		10.7.4 Why Compensating Transactions Work 
		10.7.5 Exercises for Section 10.7 
	10.8 Summary of Chapter 10 
	10.9 References for Chapter 10 

11 Information Integration 
	11.1 Modes of Information Integration 
		11.1.1 Problems of Information Integration 
		11.1.2 Federated Database Systems 
		11.1.3 Data Warehouses 
		11.1.4 Mediators 
		11.1.5 Exercises for Section 11.1 
	11.2 Wrappers in Mediator-Based Systems 
		11.2.1 Templates for Query Patterns 
		11.2.2 Wrapper Generators 
		11.2.3 Filters 
		11.2.4 Other Operations at the Wrapper 
		11.2.5 Exercises for Section 11.2 
	11.3 On-Line Analytic Processing 
		11.3.1 OLAP Applications 
		11.3.2 A Multidimensional View of OLAP Data 
		11.3.3 Star Schemas 
		11.3.4 Slicing and Dicing 
		11.3.5 Exercises for Section 11.3 
	11.4 Data Cubes 
		11.4.1 The Cube Operator 
		11.4.2 Cube Implementation by Materialized Views 
		11.4.3 The Lattice of Views 
		11.4.4 Exercises for Section 11.4 
	11.5 Data Mining 
		11.5.1 Data-Mining Applications 
		11.5.2 Association-Rule Mining 
		11.5.3 The A-Priori Algorithm 
	11.6 Summary of Chapter 11 
	11.7 References for Chapter 11