Diving Head First Into NoSQL

Used to store all of the data I collect in JSON files. NoSQL might just change that.

The Problem..

I store most of the data I generate in JSON files. They can get quite large real file, and the larger it gets, the slower it gets to read the whole file. The file is also easily accessible, so I could easily write the wrong delete command and end up losing user data.

Things I Tried..

A friend of mine used Google Firebase on an office side project for shits and giggles. I saw him use it and I thought I would give it a shot. I didn't have to install any extra software on my computer or server to use it. Fast forward a few hours and one bad line of code led me to hit my daily read limit of 50k operations in about 5 seconds...

Why I Don't Want To Use MySQL or SQLite..

Because I do not want to write raw SQL queries for my applications.

Simple as that.

I am doing most of my personal projects in Python, and most of the tutorials I've found teach MySQL using raw SQL queries, which is something I absolutely loathe to do. I haven't written any meaningful raw SQL queries in the past year ever since I started working with Laravel. Laravel's ORM is just next level.

I found a Python ORM wrapper, but that is too close to raw SQL than I'd like it to be.  To "Plan B".

Databases, Collections and Documents..

NoSQL has their own naming conventions? For anyone coming from the MySQL world, database tables are called collections, and rows in a table is called documents. Or that's at least what I have understood from reading all of W3 Shools' NoSQL tutorial. Now that I've mentioned W3 Schools, I am using MongoDB as a database for NoSQL.

The tutorials on W3 Schools is really great for anyone to get started with NoSQL. But if you want to use it programmatically in any meaningful way, you're gonna need to construct your own mini functions to handle all the CRUD operations that you'll need to do in your programs.

I just finished making my own CRUD functions. Super easy. It works with any collection and any document. Excellent.

What's Next..

Used my mini functions to test basic I/O on MongoDB. Works flawlessly.

Stumbled across a package called GridFS that lets me store files over 16MB in the database. But if anything programming has taught me is that storing files in a database is "inefficient".

Now I have a way of storing all the user data I will be storing from my Namaadhu Vaguthu Bot.

Time to upgrade all my previous projects to use NoSQL and MongoDB to store user data.