A comprehensive academic resource for Big Data Analytics (BDA) and Computational Lab - I (CL-I), featuring Hadoop ecosystem experiments, NoSQL databases, MapReduce implementations, and data mining techniques.
Overview · Contents · Reference Books · Personal Preparation · Assignments · Classwork · Quizzes · Laboratory · Mini-Project · Internal Assessment Test · Semester Exam · Question Papers · Submission Report · Syllabus · Usage Guidelines · License · About · Acknowledgments
Big Data Analytics (CSDLO7032) and Computational Lab - I (CSL704) are core subjects in the Final Year (Semester VII) of the Computer Engineering curriculum at the University of Mumbai. These courses provide foundational knowledge of Big Data technologies, distributed computing frameworks, and data analytics methodologies.
The curriculum encompasses several key domains in Big Data Analytics (BDA):
- Big Data Architectures: Hadoop Ecosystem (HDFS, YARN), MapReduce programming model.
- NoSQL Databases: Key-value stores, Column-family stores, Document databases (MongoDB), Graph databases.
- Data Mining: Frequent itemset mining (Apriori, PCY), Clustering (K-Means), Classification.
- Streaming & Analytics: Real-time data processing, Recommendation systems, Collaborative filtering.
This repository represents a curated collection of study materials, reference books, lab experiments, and personal preparation notes compiled during my academic journey. The primary motivation for creating and maintaining this archive is simple yet profound: to preserve knowledge for continuous learning and future reference.
As a computer engineer, understanding Big Data technologies is crucial for developing scalable and data-driven solutions. This repository serves as my intellectual reference point: a resource I can return to for relearning concepts, reviewing methodologies, and strengthening understanding when needed.
Why this repository exists:
- Knowledge Preservation: To maintain organized access to comprehensive study materials beyond the classroom.
- Continuous Learning: To support lifelong learning by enabling easy revisitation of Big Data concepts.
- Academic Documentation: To authentically document my learning journey through Big Data Analytics and Computational Lab I.
- Community Contribution: To share these resources with students and learners who may benefit from them.
Note
All materials in this repository were created, compiled, and organized by me throughout my undergraduate program (2018-2022) as part of my coursework, laboratory assignments, and project implementations.
This collection includes comprehensive reference materials covering all major topics:
| # | Resource | Focus Area |
|---|---|---|
| 1 | BDA Modules 1-3 | Introduction to Big Data, Hadoop, and MapReduce |
| 2 | BDA Modules 4-6 | NoSQL, Mining, and Analytics |
| 3 | BDA Modules 7-10 | Advanced Topics and Case Studies |
| 4 | Logic & Chain Rules | Fundamental Logic Concepts |
| 5 | BDA Index | Topical Index |
Study materials and planning resources for effective exam preparation:
| # | Resource | Description |
|---|---|---|
| 1 | Subject Choice | Department Level Optional Subject Choice |
Academic assignments for comprehensive learning and practice:
| # | Assignment | Format | Description | Date | Marks |
|---|---|---|---|---|---|
| 1 | Assignment 1 | Handwritten | Hadoop Ecosystem, MapReduce Architecture, and NoSQL Database Concepts | October 08, 2021 | 8/10 |
| 2 | Assignment 1 | Digital | Hadoop Ecosystem, MapReduce Architecture, and NoSQL Database Concepts | October 08, 2021 | 8/10 |
| 3 | Assignment 2 | Handwritten | Data Stream Management Systems (DSMS), Finding Similar Items, and Girvan-Newman Algorithm | October 08, 2021 | 8/10 |
Topics Covered: Hadoop Ecosystem (Components, Architecture, Limitations) · MapReduce Model (Map, Reduce, Shuffling) · NoSQL Databases (Business Drivers, Architectural Patterns) · Data Stream Management Systems (DSMS) · Finding Similar Items · Girvan-Newman Algorithm
Lab classwork sessions conducted during the online course:
| # | Resource | Description | Report |
|---|---|---|---|
| 1 | Case Study | Hadoop in Banking: The Game Changer - Big Data application in finance industry | Download |
| 2 | Case Study | Study on iPhone SE (2nd Generation): Marketing analysis and product positioning | Download |
BDA-specific quizzes conducted during the course:
| # | Quiz | Topics | Date | Marks |
|---|---|---|---|---|
| 1 | Quiz 1 | NoSQL Databases | August 18, 2021 | 6/6 |
| 2 | Quiz 2 | MapReduce Programming | August 23, 2021 | 10/10 |
The laboratory component (CSL704) focuses on hands-on implementation of Big Data technologies, including Hadoop setup, MapReduce programming, NoSQL database management, and Data Mining algorithms.
Tip
🐘 HADOOP: I specifically created this repository for this course to demonstrate Matrix Multiplication using the MapReduce paradigm. It serves as a foundational case study for mastering core Big Data Analytics concepts before proceeding with the experiments.
Prerequisites: Experiments build progressively - complete Hadoop installation and configuration (Experiments 1-3) before attempting MapReduce programming (Experiments 5, 7). Familiarity with Java is essential for Hadoop, while Python proficiency is required for PySpark experiments (Experiments 7, 9). Keep backups of your Hadoop configuration files before making changes.
| # | Experiment | Date | Report |
|---|---|---|---|
| 1 | Study of Hadoop System | October 10, 2021 | View |
| 2 | Installation of Hadoop | July 28, 2021 | View |
| 3 | Installation and Configuration of Sqoop | September 09, 2021 | View |
| 4 | Neo4j Graph Database Operations | October 10, 2021 | View |
| 5 | Matrix Multiplication using MapReduce | October 05, 2021 | View |
| 6 | Data Analysis using Bigsheets | October 05, 2021 | View |
| 7 | Word Count using PySpark MapReduce | October 05, 2021 | View |
| 8 | DGIM Algorithm Implementation | October 05, 2021 | View |
| 9 | K-Means Clustering using PySpark | October 05, 2021 | View |
| 10 | Mini-Project: Optimizing Stock Trading Strategy with K-Means Clustering | October 05, 2021 | View |
Experiment 1: Study of Hadoop System
| Document | Category | Description | Link |
|---|---|---|---|
Hadoop_Banking_Case_Study.md |
Case Study | Hadoop's Impact on Banking Sector Analysis | View |
Experiment 4: Neo4j Graph Database Operations
| Program | Category | Description | Code |
|---|---|---|---|
Neo4j_Car_Database.cypher |
NoSQL | Cypher queries for car database with nodes and relationships | View |
Experiment 5: Matrix Multiplication using MapReduce
| Program | Category | Description | Code |
|---|---|---|---|
Mapper.py |
MapReduce | Mapper for matrix multiplication (2x3 × 3x2) | View |
Reducer.py |
MapReduce | Reducer for aggregating matrix products | View |
Experiment 6: Data Analysis using Bigsheets
| Resource | Category | Description | Link |
|---|---|---|---|
CeesVee.csv |
Dataset | Truck tracking data with location and driver information | View |
CSV_Head.csv |
Dataset | Field length analysis data | View |
Experiment 7: Word Count using PySpark MapReduce
| Program | Category | Description | Code |
|---|---|---|---|
Word_Count_MapReduce.py |
PySpark | Word count implementation using Spark context | View |
Word_Count_MapReduce.ipynb |
PySpark | Interactive notebook for word frequency analysis | View |
sample.txt |
Dataset | Input text file for word count analysis | View |
Experiment 8: DGIM Algorithm Implementation
| Program | Category | Description | Code |
|---|---|---|---|
DGIM_Algorithm.py |
Stream Processing | DGIM algorithm for counting 1s in binary streams | View |
DGIM_Algorithm.ipynb |
Stream Processing | Interactive notebook for DGIM stream analysis | View |
Experiment 9: K-Means Clustering using PySpark
| Program | Category | Description | Code |
|---|---|---|---|
KMeans_Clustering.py |
Machine Learning | K-Means clustering for customer segmentation | View |
KMeans_Clustering.ipynb |
Machine Learning | Interactive notebook with silhouette score visualization | View |
CC GENERAL.csv |
Dataset | Credit card customer data (9K cardholders) | View |
Experiment 10: Mini-Project (Stock Trading Strategy)
| Document | Category | Description | Link |
|---|---|---|---|
BDA_MINI-PROJECT_REPORT_BE-COMPS_B-50,51,58.pdf |
Report | Complete project report with methodology and results | View |
BDA_MINI-PROJECT_PPT_BE-COMPS_B-50,51,58.pdf |
Presentation | Final project presentation slides | View |
| # | Resource | Description |
|---|---|---|
| 1 | Lab README | Detailed navigation guide with experiment descriptions |
Data-Driven Stock Market Analysis using Unsupervised Machine Learning
Important
Special thanks to Mega Satish and Hasan Rizvi for their meaningful contributions, guidance, and support that helped shape this work.
This mini-project explores the application of K-Means Clustering, an unsupervised machine learning algorithm, to optimize stock trading strategies. By analyzing historical stock data, the system groups stocks based on performance metrics (volatility, returns), enabling investors to identify patterns and make data-driven decisions. The project leverages Python's data analysis libraries (Pandas, NumPy, Scikit-learn) to perform clustering and visualize market trends.
| # | Resource | Description | Date |
|---|---|---|---|
| 1 | Project Repository | Complete source code and documentation | - |
| 2 | Project Report | Detailed documentation and analysis | October 05, 2021 |
| 3 | Presentation | Visual overview of the methodology | October 05, 2021 |
| 4 | Source Code | Jupyter Notebook implementation | - |
Internal assessment evaluations conducted during the course:
| # | Resource | Description |
|---|---|---|
| 1 | Answer Sheet | BDA Internal Assessment Test 1 Answer Sheet |
| # | Resource | Description |
|---|---|---|
| 1 | Answer Sheet | BDA Internal Assessment Test 2 Answer Sheet |
Important
COVID-19 Impact: This coursework was completed during the COVID-19 pandemic. All examinations and assessments were conducted in a digital format.
Final semester examination submission:
| # | Resource | Description | Date |
|---|---|---|---|
| 1 | Answer Sheet | BDA Semester Exam Answer Sheet | November 29, 2021 |
University of Mumbai examination papers from 2012-2019:
| # | Exam Session | Syllabus | Resource |
|---|---|---|---|
| 1 | May 2019 | CBCGS | View |
| 2 | December 2018 | CBCGS | View |
| 3 | May 2018 | CBCGS | View |
| 4 | December 2017 | CBCGS | View |
| 5 | May 2017 | CBCGS | View |
| 6 | December 2016 | CBCGS | View |
| 7 | May 2016 | CBCGS | View |
| 8 | December 2015 | CBGS | View |
| 9 | May 2015 | CBGS | View |
| 10 | December 2014 | CBGS | View |
| 11 | May 2014 | CBGS | View |
| 12 | December 2013 | CBGS | View |
| 13 | May 2013 | CBGS | View |
| 14 | December 2012 | CBGS | View |
| 15 | May 2012 | CBGS | View |
Course completion documentation:
| # | Document | Description |
|---|---|---|
| 1 | Submission Report | Final coursework submission report |
Official CBCGS Syllabus
Complete Final Year Computer Engineering syllabus document from the University of Mumbai, including detailed course outcomes, assessment criteria, and module specifications for BDA and BDA Lab.
Important
Always verify the latest syllabus details with the official University of Mumbai website, as curriculum updates may occur after this repository's archival date.
This repository is openly shared to support learning and knowledge exchange across the academic community.
For Students
Use these resources as reference materials for understanding Big Data architectures, data analytics algorithms, and preparing for examinations. All content is organized for self-paced learning.
For Educators
These materials may serve as curriculum references, lab examples, or supplementary teaching resources. Attribution is appreciated when utilizing content.
For Researchers
The documentation and organization may provide insights into academic resource curation and educational content structuring.
This repository and all linked academic content are made available under the Creative Commons Attribution 4.0 International License (CC BY 4.0). See the LICENSE file for complete terms.
Note
Summary: You are free to share and adapt this content for any purpose, even commercially, as long as you provide appropriate attribution to the original author.
Created & Maintained by: Amey Thakur
Academic Journey: Bachelor of Engineering in Computer Engineering (2018-2022)
Institution: Terna Engineering College, Navi Mumbai
University: University of Mumbai
This repository represents a comprehensive collection of study materials, reference books, assignments, and personal preparation notes curated during my academic journey. All content has been carefully organized and documented to serve as a valuable resource for students pursuing Big Data Analytics.
Connect: GitHub · LinkedIn · ORCID
Grateful acknowledgment to Mega Satish for her exceptional contribution to this repository through the collaborative Mini Project on Stock Trading Strategy Optimization with K-Means Clustering. Her analytical insights, technical expertise, and dedication to exploring Big Data analytics applications in financial markets made a real difference. Working alongside Mega was transformative, not only because of her strong grasp of machine learning algorithms, but because she truly embraced the collaborative spirit of learning. Her thoughtful approach to problem-solving, openness to experimentation, and steady encouragement turned challenges into meaningful learning moments. This work reflects the growth that came from learning side by side. Thank you, Mega, for everything you contributed along the way.
Grateful acknowledgment to Hasan Rizvi for his exceptional contribution to this repository through the collaborative Mini Project on Stock Trading Strategy Optimization with K-Means Clustering. Working alongside Hasan was a truly enriching experience - his technical expertise in data preprocessing and clustering algorithms, creative problem-solving, and unwavering dedication made every challenge feel surmountable. Beyond his analytical skills, it was his collaborative spirit and enthusiasm for exploring Big Data applications that transformed this project from a concept into a robust utility. His patience in code reviews, willingness to tackle complex algorithms, and commitment to delivering quality work inspired growth and learning throughout the development process. Thank you, Hasan, for being an outstanding project partner and for all the knowledge you shared along the way.
Grateful acknowledgment to the faculty members of the Department of Computer Engineering at Terna Engineering College for their guidance and instruction in Big Data Analytics. Their clear teaching and continued support helped develop a strong understanding of distributed systems and data analytics.
Special thanks to the mentors and peers whose encouragement, discussions, and support contributed meaningfully to this learning experience.
Overview · Contents · Reference Books · Personal Preparation · Assignments · Classwork · Quizzes · Laboratory · Mini-Project · Internal Assessment Test · Semester Exam · Question Papers · Submission Report · Syllabus · Usage Guidelines · License · About · Acknowledgments
🔬 Computational Lab I · 📈 Stock Trading Optimization with K-Means · 🐘 HADOOP
Computer Engineering (B.E.) - University of Mumbai
Semester-wise curriculum, laboratories, projects, and academic notes.