Large Scale Information Storage and Retrieval: DS 4300 – Fall 2024

• DS 4300 – Fall 2024 – Course Syllabus
• TA Office Hours **New**
• Campus Wire for 4300
• GradeScope for 4300

• Lecture 01 – Introduction
• Module 02 – Data Structure’s Effects on Performance
  • Note the additional references added to the Playlist on O’Reilly. I’ve added sections from books on BSTs and AVL Trees specifically in Python.
  • Binary Search Trees References:
    • Notes from U Rochester (Stop at page 20)
    • UCI Notes on AVL Trees (Stop at Analysis section)
  • AVL Tree Rotations Diagram
  • B+ – Trees
    • Open Data Structures – B-Trees and B+-Trees (may need the ref the material above this part of the page)
    • Notes on B-Trees (but really they are B+ Trees)
    • In Class B+ Tree Practice
• Module 03 – The Limits of the Relational Model
• Module 04 – Replication and Partitioning – Part 1
• Module 05 – Replication and Partitioning – Part 2
• Module 06 – Document DBs + MongoDB
  • Youtube: ScriptBytes – How to run MongoDB in Docker – Follow Along
    • When using the VS Code Plugin to test the connection, you don’t need to type out all the different JSON data he puts in the Javascript code. It is simply to confirm the connection is working.
    • Pay close attention to the last section on Docker Compose
  • Youtube: NeuralNine – Connecting to Mongo with PyMongo
    • This link is set up to start around 7:16 into the video. The first part of the video is about installing MongoDB, which you would have done from the previous video.
  • Your goal: using Python, insert 5 of the JSON documents (news articles) from A01 into your MongoDB instance using PyMongo. This will require going through the videos as well as a bit of independent research. Since this isn’t a HW assignment technically, you’re more than welcome to work together. A02 will build off of these topics, though.
• Module 07 – MongoDB – Part 1
  • Mongo Intro Jupyter Notebook.
  • Mongo Aggregation Jupyter Notebook
• Module 08 – KV Databases and Redis
• Module 09 – Redis Interactive CLI
• Module 10 – Redis Data Types/Structures (Top 4)
• Module 11 – Graphs – Intro to Graph DBs
• Module 12 – Neo4j. Neo4j – Building Relationships
• Module 13 – AWS Let’s Go!
• Module 14 – AWS – MySQL DB
• Module 15 – Big Data
• Module 16 – S3
• Module 17 – EC2 and Lambdas

• Assignment 1 – Indexing Structures
  • Conceptual Component – Handout
  • Implementation Component – GH Classroom Link
    • Download Dataset
  • EC Due Date: Sept 22 @ 11:59pm
  • Regular Due Date: Sept 24 @ 11:59pm
• Assignment 2 – Jupyter Notebook
  • EC Due Date: Oct 16 @ 11:59 pm
  • Regular Due Date: Oct 18 @ 11:59 pm
• Assignment 3
  • EC Due Date:
  • Regular Due Date:
• Assignment 4
  • EC Due Date:
  • Regular Due Date:
• Assignment 5
  • EC Due Date:
  • Regular Due Date:

Template Repository for Project

• Project Handout – Due Dec 7, 2024 @11:59pm EST

Additional Helpful Information:

• Connect EC2 and RDS in Webapp
• AWS RDS Tutorials
• Python Boto3 Library – Python library to interact with AWS services
• More to come…

• Python Style Guide (PEP 8)