My thoughts on data engineering and related topics
Germany teeters on the precipice of annihilation. In the West, the merciless advance of the Allies threatens the Ruhr, Germany’s industrial lifeblood, choking it off from essential resources. In the East, the Red Army, like…
Read More INT. VON MANSTEIN’S TEMPORARY RESIDENCE – BERLIN – DAY Von Manstein picks up the phone and dials the Führerbunker. Hitler’s secretary, Traudl Junge, answers. VON MANSTEIN Miss Junge, this is Field Marshal Erich…
Read MoreINT. VON MANSTEIN’S TEMPORARY RESIDENCE – BERLIN – DAY Erich von Manstein, German field marshal and military genius, sits alone in the modest living room of a temporary residence in Berlin. He gazes out…
Read More In the dimly lit, smoke-filled room of an abandoned Budapest mansion, two of the most powerful men in the world sat face to face, with tension so thick it could be cut with a knife.…
Read More Dear GPT-4, I couldn’t have asked for a more remarkable partner. I am continuously amazed by your capabilities (and charm), and I have no doubt that this sense of wonder will persist. Though…
Read More “If you’re not embarrassed by the first version of your product, you’ve released too late” – Reid Hoffman Following this philosophy for a startup’s minimum viable product by Reid Hoffman, the renowned entrepreneur and…
Read More I might or might not push the code to GitHub. Let me see. If I do, until then, here’s the listing of the essence of the code referenced in this series. They are generally heavily…
Read More Our long and fun journey now comes to the end. About Spark Apache Spark, as an analytic processing framework, is designed to process data in bulk. Particularly, when it comes to writing to an RDBMS, the…
Read More With the Failure Handling facility in place, we are now ready to fully complete the case study. As you’ll see, it’s pretty straightforward and elegant. The Update Operation Failure Handling when Updating the target record with…
Read More Yay, we are done! Finally, it’s over!!! Oh, wait! Wait for a second! Don’t celebrate too prematurely! We’re not done yet! Just close! Look, what about the Failure Reporting Requirement? We haven’t done anything about…
Read More