Glen Takahashi

Super interesting (and technical) breakdown on how the folks at ClickHouse are transforming the simple, flexible JSON format into something that can be leveraged at hyperscale. An excellent example of how sometimes scale can make you do crazy things.

1

Last Tuesday, Malsgufa, our legacy CMS backend, was shut down. This milestone for BI's Tech and Product team culminated years of planning, effort, and execution. Malsgufa's first commit was on July 25, 2014, leading to 5,186 commits across nearly 30 repositories. Built and maintained by scores of engineers, its deprecation represents significant cost savings (20 production replicas, 3 instances of Solr, and an insane number of Zookeeper instances!), reduced security risk, and the elimination of an often-blamed boogeyman. The journey began in January 2020. Our challenge: moving traffic off Malsgufa without rebuilding everything in Muse. Initially, we tried GraphQL as an intermediary between Viking and our services, but progress was limited by Viking's complex Angular/Vue architecture. In January 2023, we pivoted to the strangler fig pattern with a shim. This approach proved far more effective – all Malsgufa traffic went through the shim first, with verbose logging to track unhandled requests. Meanwhile, we built Story Versioning API (SVAPI), incorporating lessons learned from Malsgufa's limitations around history and publishing states. We ran four major migrations (stories, custom pages, slideshows, and videos), building support in Muse and shimmed compatibility for Viking. Traffic gradually decreased from a torrent to a trickle, leading to the gloriously uneventful shutdown. Next, we will deprecate Viking by Q2 2025, unifying our users in the Muse ecosystem to create the industry's most innovative CMS. Special thanks to the entire Story Creation team: Chase Gruber, Chris Speziale, Daniel Arita, Danny Feliz, Dasha Lary, Dawin Camillo, Eric Saam, Frederica Chen, John Laslo, Kelly Filush, Morgan Cohn, Sarah Pai, Tyler Greenfield! Thanks also to DaD, Engagement, Helpfulness, Cloud Engineering, Test Engineering, and Operational Excellence! Finally, we recognize Malsgufa itself. Developed as a unified CMS backend to support Business Insider's expansion into new markets, it served our global newsroom reliably for a decade, publishing millions of stories despite no major changes in recent years. While it became less sustainable as our business evolved, it deserves our respect. Thank you, Malsgufa. May you rust in peace.

41

6 Comments

When the function calling paradigm is being built into other SDKs (via the Stripe toolkit.getTools() call in this case), you're seeing new de-facto standards emerge. The interfaces between LLMs and other systems are a leading indicator of how services view their business and technical relationship with model providers.

5

This is a cool project, and another example of the emerging "replace stateful storage with S3" trend. I do wonder at what point someone will package this sort of thing up into a FUSE filesystem to make it generally usable across existing services. Or maybe every service has unique enough needs that it's better to handle it custom each time. https://lnkd.in/etcVBfVf

9

I have a new article about how to use OpenTelemetry in Event Driven Architectures (more specifically with Kafka and NestJS): - Medium: https://lnkd.in/gA8UgPmE - Hashnode: https://lnkd.in/gHmAReNj

7

Come join me and my amazing colleague Steve Cole for our chalk talk at re:Invent! It'll be at 9am on Monday, Dec 2 at the Wynn, so grab your coffee and come join us for what I expect to be a lively, interactive discussion! We'll be talking Auto Scaling Groups, Spot, and everything in between. Here's our abstract: "Amazon EC2 Auto Scaling helps you maintain application availability and enables you automatically add or remove EC2 instances using Scaling policies that you define. In this chalk talk, deep dive into Amazon EC2 Auto Scaling innovations and best practices designed to increase the resiliency of your applications while meeting your cost-efficiency objectives. Learn how your deployment patterns and tools such as Amazon EC2 Auto Scaling instance maintenance policy and instance refresh rollback can help you achieve the exact balance of healthy compute capacity needed." #reinvent #reinvent2024 #cmp202 #autoscaling #ec2 #ec2spot

74

3 Comments

One of the coolest things we've been building at Truewind is our document parsing and extraction engine.   A lot of accountants deal with scanned or photographed documents like receipts, contracts, invoices, and bills.   For us, the goal is to make sifting through and classifying that information a lot easier.   Matthew Duong has been leading the charge on this project, and recently wrote about the experience on our engineering blog. In particular, he explains why ditching schemas and using PostgreSQL’s JSONB has made such a big difference for our platform.   If you’re interested in the technical side of how we’re refining accuracy and speed in document processing, definitely give it a read.   Here’s the link: https://lnkd.in/gN3Sgext

16

I started writing regularly about AI. If you're an innovators or product developer eager to apply the latest in AI, whether you're in a startup or a big company, consider checking https://lnkd.in/gGdW8PYe I won't teach you ML, but I share real-world insights from launching - and sometimes failing at - numerous ML features on a large scale.

54

2 Comments

Check out our latest Tech Talk, sharing how we build systems for growth at Faire, touching on our move to a monorepo, how we use Next.js & our automation bot Fairey! Watch the full video in this blog post https://lnkd.in/gbkGsVNU

45

1 Comment

Do you think you need to hire developers as w2 employees to build a highly-skilled, devoted and culturally aligned team? Let Blueprint show you otherwise. Here's how we're changing the game: 1️⃣ Unmatched Quality: Quality is our creed. Our vetting process includes interviewing with former CTOs and senior software engineers, ensuring only top-tier talent makes it to your team. 2️⃣ Long-Term Commitment: Forget the turnover and churn worries! Our developers stay longer than industry norms for W2 developers (average tenure of 2.2 years), providing stability and continued expertise to your projects. 3️⃣ Cultural Integration: Culture isn't just an add-on for us. We dive deep to understand your core values and ensure every developer we match with your team enhances and embraces your company culture. 4️⃣ Equity is Optional: Save your cap table for your growing startup! We ensure that our developers are passionate believers in your product right from the start - without incentivizing them with equity. We rigorously test for craftsmanship, ensuring they value not just a quality end result but also the journey of creation. 🌟 Shoot us a DM to connect and see how we can improve your staffing strategy. #StaffAugmentation #InnovationInStaffing #TechTalent #CultureFit #LongTermGrowth

20

1 Comment

Here's my first new talk in years, grown from one of the seeds Dragan Stepanović is always planting. It's for anyone who cares about software investments yielding returns. It's called "Not So #ExtremeProgramming": https://lnkd.in/ebywkddq #XP #XPLives #Agile #Scrum #LeSSIsMore

25

1 Comment

🏎️💨 Making of a Ridiculously Fast™ API client in R. Why my package is **~17x faster** than the community alternatives that call the **exact same API endpoints**. 👇🏽**design choices matter** ✨ https://lnkd.in/ggi_2XyQ

129

2 Comments

After a year of work my book Building A Startup - A Primer For The Individual Contributor is finally available! https://lnkd.in/erThxvpP If you’re working in or around startups as an IC or technical leader this book is for you. It’s based on 20 years of startup work and will help set you up for success to do more with less, move faster and more correct, and spend more time building features and product people love and less time banging your head at roadblocks and bottlenecks.

115

12 Comments

We've been making some cool examples of things that can be built with substrate.run lately. Here's a simple chatbot that can help you cook: https://lnkd.in/eEZm-nRJ If you're working on building a chatbot, take a look and see what it might look like to use Substrate 🧑‍🍳

19

2 Comments

tldr: I'm hiring! As many of you know, I recently joined Lattice as their SVP of Engineering and I'm excited to announce that I'm hiring for all Engineering levels and more broadly, Lattice has open roles across EPD. If you're wondering, why Lattice? The last few years have seen a step change in how we think about the meaning of work and the employer <> employee relationship - add on the fuel of AI taking our jobs and you could be forgiven for thinking the future of work looks bleak for employees. It doesn't need to be that way - here at Lattice we're building the next generation of talent tools to give employers insights and employees purpose because people matter at work. Among other things, we're using generative AI to securely analyze enormous volumes of data to unlock career insights faster and investing in building equitable feedback, compensation and employer analytics tools - not just for one company but for all companies. If you're interested in making a difference to the future of work then let's chat - all roles are remote eligible and based in the US. We have physical offices in San Francisco and NYC if you prefer the in-office environment.

323

48 Comments

I believe it is critical to admit when we're wrong. And folks, I was wrong in this post: https://lnkd.in/gGWhWwDW Today I had the opportunity to chat with Andrew Regenscheid from National Institute of Standards and Technology (NIST) about NIST SP800-63B rev4 draft, and specifically the guidance on synced passkeys. During our chat I shared my understanding of the NIST guidance based on my reading of the document. Andrew helped clarify the intent, which was not how I and others read the draft guidance. Per our discussion, credential service providers (CSPs) should consider synced passkeys as AAL2 compatible credentials. We talked about how I and others arrived at the conclusion that synced passkeys did not all meet this bar, and I shared thoughts on how NIST can clarify their writing to establish a common understanding of the guidance. Thank you Andrew for taking the time to chat, helping me understand NIST's perspective, and being open to constructive feedback.

35

6 Comments

Today, we’re publishing our biggest piece yet.   We partnered with Puzzle to write about the next generation of accounting tools.   Our CEOs, Alex Lee and Sasha Orloff, explain how AI can turn accounting from a repetitive chore into a major source of business intelligence.   They cover everything from decreasing costs to improving accuracy and enhancing customer support.   We also hear from Fouad Farhat of Soma Capital—an investor in both Truewind and Puzzle—about the current state of accounting, and where it’s headed.   📌Check out the full article here: https://lnkd.in/eUzvZjTm

22