D4M: Signal Processing on Databases

Follow D4M: Signal Processing on Databases

Share on

This course focused on the signal processing on databases, based on detection theory and linear algebra with databases. By bringing together the concepts of strings and searching, student applied the big data analysis skills to new domains.

Jeremy Kepner

Nov 9, 2018 LATEST EPISODE
infrequent NEW EPISODES
38m AVG DURATION
18 EPISODES

Search for episodes from D4M: Signal Processing on Databases with a specific topic:

Latest episodes from D4M: Signal Processing on Databases

Lecture: Mathematics of Big Data and Machine Learning

Play Episode Listen Later Nov 9, 2018 38:16

Jeremy Kepner talked about his newly released book, "Mathematics of Big Data," which serves as the motivational material for the D4M course.

big data lecture machine learning mathematics

Lecture 0: Introduction

Play Episode Listen Later Jun 6, 2016 50:28

Introduction to signal processing applied to graphs. Course outline. Discussion of relevant technologies programming and storage technologies. Constructing a graph from raw data.

lecture constructing

Lecture 1: Examples Demonstration

Play Episode Listen Later Apr 13, 2016 24:57

D4M.mit.edu software demo example/1Intro/2EdgeArt. Adjacency matrix construction. Incidence matrix construction.

lecture demonstrations incidence

Lecture 3: Entity Analysis in Unstructured Data

Play Episode Listen Later Apr 13, 2016 45:42

Historical evolution of the web and cloud computing. Using the exploded (D4M) schema. Analyzing computer network data. Analyzing computer network data.

historical lecture analyzing entity unstructured data

Lecture 4: Analysis of Structured Data

Play Episode Listen Later Apr 13, 2016 41:53

Computing statistics and analytics on data in the exploded (D4M) schema.

lecture computing structured data

Lecture 6: Examples Demonstration

Play Episode Listen Later Apr 13, 2016 22:22

D4M.mit.edu software demo examples/2Apps/4BioBlast. Distribution of genetic sample data. Ingesting genetic data into an associative array using the exploded (D4M) schema. Correlating genetic data via associative array multiplication.

lecture distribution demonstrations ingesting correlating

Lecture 3: Examples Demonstration

Play Episode Listen Later Apr 13, 2016 32:56

D4M.mit.edu software demo example/2Apps/1EntityAnalysis. Incidence array of text entities. Computing the entity degree distribution.

lecture demonstrations computing incidence

Lecture 6: Bio Sequence Cross Correlation

Play Episode Listen Later Apr 13, 2016 44:52

Genectic sequence analysis using associative arrays. Creating of a genetic processing pipeline. Ingest of genetic sample data into a database. Sub-sampling of data. Correlation of genetic samples using associative array multiplication.

cross lecture sequence correlation ingest

Demonstration 7

Play Episode Listen Later Apr 13, 2016 83:22

D4M.mit.edu software demo examples/3Scaling/2ParallelDatabase. MIT SuperCloud database management system. Starting the Apache Accummulo database. High performance database ingest. Using the D4M schema.

starting demonstrations

Lecture 7: Kronecker Graphs, Data Generation, and Performance

Play Episode Listen Later Apr 13, 2016 44:42

Theory of Kronecker graphs. Database ingest performance and database query performance. Array multiplication performance.

performance data generation theory lecture databases graphs array kronecker

Lecture 7: Examples Demonstration

Play Episode Listen Later Apr 13, 2016 13:17

D4M.mit.edu software demo examples/3Scaling/1KroneckerGraph. Generation of power law graphs via Kronecker products. D4M.mit.edu software demoexamples/3Scaling/3MatrixPerformance. Measuring the performance of array multiplication.

generation measuring lecture demonstrations kronecker

Lecture 2: Group Theory

Play Episode Listen Later Apr 13, 2016 71:42

Associative array mathematics. Relevant operations on an associative array. Semirings and matrices. See MIT Press book "Mathematics of Big Data."

relevant big data lecture mathematics associative group theory

Lecture 2: Examples Demonstration

Play Episode Listen Later Apr 13, 2016 7:15

D4M.mit.edu software demo example/1Intro/3GroupTheory. Associativity, commutativity, distributivity properties.

lecture demonstrations associativity

Lecture 5: Perfect Power Law Graphs -- Generation, Sampling, Construction, and Fitting

Play Episode Listen Later Apr 13, 2016 53:49

Statistical distribution of background/noise in databases. Power law distribution describes many backgrounds. Perfect power law distribution can be used to bin and model the background data.

power generation construction lecture fitting graphs sampling statistical power law

Lecture 5: Examples Demonstration

Play Episode Listen Later Apr 13, 2016 24:05

D4M.mit.edu software demo examples/2Apps/3PerfectPowerLaw. Generate synthetic power law distributions. Analyze power law distributions in real data.

lecture generate demonstrations analyze

Lecture 1: Using Associative Arrays

Play Episode Listen Later Apr 13, 2016 45:24

Creating an exploded database schema. Standard database processing chain. Graph adjacency matrix. Vertex degree distribution. Directed graphs, multi-graphs, and hyper-graphs. Graph incidence matrix.

directed lecture graphs array vertex associative

Lecture 4: Examples Demonstration

Play Episode Listen Later Apr 13, 2016 23:59

D4M.mit.edu software demo example/2Apps/2TrackAnalysis. Tracking entities through space and time.

tracking lecture demonstrations

Lecture 0: Examples Demonstration

Play Episode Listen Later Apr 13, 2016 31:55

Introduction to associative arrays. D4M.mit.edu software demo example/1Intro/1AssocIntro. Creating, writing, reading, selecting, and performing math on associative arrays.

lecture demonstrations

Claim D4M: Signal Processing on Databases

In order to claim this podcast we'll send an email to with a verification link. Simply click the link and you will be able to edit tags, request a refresh, and other features to take control of your podcast page!

Claim Cancel