Group items tagged

It's officially winter, so what could be better than drinking hot chocolate while querying the new Stack Overflow dataset in BigQuery? It has every Stack Overflow question, answer, comment, and more - which means endless possibilities of data crunching. Inspired by Felipe Hoffa's post on how response time varies by tag, I wanted to look at the comments table (53 million rows!).

Web/app projects these days often have many distributed parts. It's not uncommon for groups to use the right tool for the job. The right tools are often something like the choice below. Redis for queuing, and caching. Elastic Search for searching, and log stash. Influxdb or RRD for timeseries. S3 for an object store. PostgreSQL for relational data with constraints, and validation via schemas. Celery for job queues. Kafka for a buffer of queues or stream processing. Exception logging with PostgreSQL (perhaps using Sentry) KDB for low latency analytics on your column oriented data. Mongo/ZODB for storing documents JSON (or mangodb for /dev/null replacement) SQLite for embedded. Neo4j for graph databases. RethinkDB for your realtime data, when data changes, other parts 'react'. ... For all the different nodes this could easily cost thousands a month, require lots of ops knowledge and support, and use up lots of electricity. To set all this up from scratch could cost one to four weeks of developer time depending on if they know the various stacks already. Perhaps you'd have ten nodes to support. Could you gain an ops advantage by using only PostgreSQL?

I was playing a bit with OrientDB because the licensing model is not that pricey as for Neo4j. Anyway, after having really bad experience (http://orientdbleaks.blogspot.com/) I returned back to CouchDB, although it's not a graph db. But while I was searching for more data, I found this: http://www.aptuz.com/blog/is-postgres-nosql-database-better-than-mongodb/ and this: https://www.arangodb.com/2015/10/benchmark-postgresql-mongodb-arangodb/ so I'm pondering with an idea to give postgres(no)sql a chance :)