When your “code” is a neural network with billions of parameters, traditional debugging tools barely scratch the surface.

So, the real challenge has shifted from fixing deterministic code to making sense of probabilistic behavior.

To explore this, we spoke with Zvonimir Petkovic, Senior Software Engineer at Infobip.

AI-assisted debugging – helping hands or crutches?

When asked how AI is reshaping debugging, and whether it’s making developers sharper or just more reliant on automation, Zvonimir says it all comes down to mindset: it depends on whether a developer has truly embraced AI coding in a “vibe” way or not.

Regardless, AI tools often make debugging easier – even if you never tell Cursor to “pls fix this.”

But when AI is doing more of the heavy lifting in debugging, what happens to a developer’s critical thinking? Here’s how Petkovic sees it:

It comes down to mindset: for most people, the process offers a learning opportunity, but whether they actually benefit depends on having the time and will to dig in – or whether the project’s pace leaves no room for deeper exploration.

Why AI struggles with legacy code

When asked about the biggest challenges of using AI for debugging – particularly around accuracy and understanding context – Zvonimir points to one major issue: legacy projects.

Greenfield projects are easy, but try debugging a years-old, messy, poorly documented codebase, and it quickly turns into a nightmare.

Petkovic also noted that AI models have limits when it comes to context – they can’t always understand everything at once. That’s why context engineering is more crucial than ever.

We also asked Zvonimir how junior developers fare with AI-assisted debugging compared to senior engineers – does it speed up their learning, or risk slowing down their skill development? He explained:

GenAI has proven a major boost for juniors or developers transitioning to a new stack. In most cases, it helps them learn faster, without having to call a senior every time they hit a problem.

The dark side of AI autonomy? Bugs and tech debt

Debugging isn’t just about fixing bugs. It’s also about keeping code secure and products reliable. Zvonimir pointed out the risks of leaning too heavily on AI to automatically fix problems.

AI-generated code can quickly become a mess and a major source of technical debt if left unchecked and the same goes for automated bug fixes, especially when you push AI autonomy beyond what’s safe today.

He also warned about the risks of a Git-connected AI agent that automatically merges PRs and performs code reviews without a human in the loop. Today’s models are much more capable, but still far from fully autonomous.

GenAI coding assistants won’t replace your principal engineers

Finally, Petkovic shared his thoughts on whether AI can really grasp the intent behind code well enough to spot deeper logic or architectural bugs:

With enough context (code and docs) GenAI assistants won’t replace your principal engineers, but they can help them see problems from new angles and iterate faster.

Looking ahead, one big question is the balance between human and AI-driven debugging: could we ever reach a point where the process is fully automated?

It’s heading in that direction. GitHub is rolling out more AI-driven debuggers that not only triage problems based on code and comments but also attempt to propose solutions.

“With GenAI models becoming better and better, and with context improving with different sources expandable by the MCP, I see these becoming just better over time. Still, it’s hard to say how quickly they will be totally independent, even though some major companies are allowing this sort of action in isolated projects,” Zvonimir concluded.

The post Debugging in the Age of AI Isn’t About Fixing Broken Code appeared first on ShiftMag.

Without proper cryptography, the answer is often yes.

It doesn’t take much for attackers – an exposed API, a weak hash, or sloppy design can open the door to stolen data, cracked passwords, or injected queries. The risks are real, and the cost of neglect is steep.

That’s why cryptography isn’t a bonus feature. In this article, we’ll dive into the different kinds of encryption you’ll encounter, and walk through how to weave these techniques into your Node.js applications in practical, secure ways – together with Yonatan Mevorach, Developer Advocate at Wix.

The different kinds of encryption and what they do

Think of cryptography as secret-keeping with math – the art of turning plain information into puzzles only the right key can solve. It’s built on a few core principles that Yonatan emphasizes:

• Confidentiality: Ensuring that information is only accessible to authorized parties.
• Integrity: Guaranteeing that data remains unaltered during transmission or storage.
• Authentication: Verifying the identity of users or systems.
• Non-repudiation: Preventing entities from denying actions they have taken.

These principles shape how secure systems are built and come to life through encryption and cryptographic algorithms. At its core, encryption takes readable data and scrambles it into ciphertext, keeping it safe from prying eyes. In his lecture at the Heapcon conference, Yonatan outlines the main approaches commonly used in modern development:

1. Symmetric Encryption: Uses a single key for both encryption and decryption. Examples include AES (Advanced Encryption Standard) and DES (Data Encryption Standard)
2. Asymmetric Encryption: Uses a pair of keys (public and private) for encryption and decryption. Standard algorithms include RSA (Rivest-Shamir-Adleman) and ECC (Elliptic Curve Cryptography)
3. Hashing: Converts data into a fixed-length hash value, which is irreversible. Examples: SHA-256 (Secure Hash Algorithm), Bcrypt, and Argon2 for password hashing.
4. Hybrid Encryption: Combines symmetric and asymmetric encryption to balance performance and security. A common implementation is TLS (Transport Layer Security).
5. Key Derivation Functions (KDFs): Used to derive strong cryptographic keys from passwords or other input values. Examples include PBKDF2 (Password-Based Key Derivation Function 2), Argon2 (Winner of the Password Hashing Competition), and Scrypt (Designed to resist hardware-based attacks).

KDFs are crucial for securely storing passwords and generating encryption keys that are resistant to brute-force attacks.

Securing your apps with Node.js, crypto modules, and modern encryption

As Mevorach explains, developers integrate encryption and cryptographic techniques into applications across various domains:

• Secure Communications: Protocols like SSL/TLS secure web traffic and APIs.
• Data Protection: Encrypted databases and file systems (e.g., AES-encrypted storage in cloud environments).
• Authentication & Access Control: Multi-factor authentication (MFA), digital signatures, and cryptographic tokens.
• Blockchain & Cryptocurrencies: Cryptographic hashing secures transactions and smart contracts.

Role of the Crypto Module in Node.js

In Node.js, the crypto module provides a set of cryptographic functionalities, including hashing, encryption, and key generation, ensuring to read the official documentation for detailed API usage.

It is widely used in web applications, authentication systems, and secure communication protocols.

Developers can leverage the crypto module to implement secure password storage, encrypt sensitive data, and integrate robust security mechanisms into Node.js applications.

Diffie-Hellman Key Exchange:

This solution solves the key distribution problem by allowing two parties to establish a shared secret over a public channel.

The process? User 1 and User 2 exchange public keys derived from private keys. Each uses the other’s public and private keys to compute the shared secret. Implementation in Node.js uses “crypto. create Diffie-Hellman.”

Developers can protect user data!

“Security best practices are simple in principle but critical in execution. Use proven algorithms like AES-256 and RSA, derive keys securely, and avoid weak or guessable passwords. Always rely on cryptographically secure random generators.”

That said, cryptography comes with its own challenges: key management, performance overhead, and evolving threats. Still, if you follow these best practices, you’re giving your software and your users real protection:

• Use well-established cryptographic libraries (e.g., OpenSSL, Bouncy Castle).
• Regularly update algorithms to mitigate vulnerabilities.
• Implement proper key management (e.g., Hardware Security Modules, cloud-based KMS solutions).
• Apply encryption at rest, in transit, and during processing where necessary.

So, as cyber threats become more sophisticated, encryption remain tools for secure software development. By integrating strong encryption practices and following industry standards, developers can build trustworthy systems that protect user data and maintain system integrity.

The post Can Anyone See This Password? Probably Yes – Unless You Encrypt It appeared first on ShiftMag.

One Monday afternoon, I received a call from my manager, which went along these lines: “Hi Dan, how are you? Would you like to travel to the US on Wednesday and take over some legacy services?“. I had a lot of questions and not a lot of time to discuss everything, but I still agreed. This call changed what I would be working on for a year or more. 

The next day, I got more context about the work and our end goal. It was to learn enough about the legacy system to keep it running until the replacement system was built and customers migrated. One caveat was that we had just 2 months to learn from the engineers working on it, so it was a tight schedule.  

Fast forward one year, we have built a completely new system and are migrating customers. In the meantime, the legacy system was still functioning, and we hadn’t experienced any major issues. Not that there weren’t issues or a few surprises along the way, but we must have done something right as we managed to go from zero to enough knowledge about the system and how to maintain it in less than two months. 

Getting the lay of the land 

I knew next to nothing about the domain and workload the system was built to automatize. Luckily for me, most of it was written in Java, so I didn’t have to learn a new language as well as a new domain.  

We organized a few in-person days with the current engineering owners, business stakeholders, and internal users to get a good start. There, we got an overview of everything and started diving into the details of both the processes and implementations of various services.  

This is when we realized that we were really dealing with two subsystems. One consisted of two large monoliths over a decade old; the other was a newer microservices-oriented architecture. The latter was supposed to replace the two monoliths, but it was being done feature by feature. This meant that both subsystems were handling user’s requests, and we had to master both of them. 

We started by collecting the important information about each service involved in handling the user’s requests. That consisted of high-level container diagrams showing most of the services, databases, and queues and how they communicated with each other.  

For each service, we gathered the locations of the following: 

• Source code 
• Configuration files 
• Build jobs 
• Deployment jobs 
• Any other useful links or documentation 

When we were done, there were 17 services and 5 databases to take over. Although it seems like you could search for that information when needed, having everything in one place proved very useful, especially later on when we needed to onboard new people. Also, knowing where everything is located meant we could check that we have permissions for the tools and access to the necessary virtual machines and databases. 

Rolling up our sleeves 

While waiting for all the permissions mentioned above, it was time to do what we did best: read the code.  

During the sessions with the business stakeholders, we gained an understanding of which functionalities were more important and which ones could be done manually if needed. That enabled us to first cover the ones that bring the most value.  

We focused on happy paths because going through every line of the code wasn’t an option. I have to say that those monolith services were quite easy to read, as everything important was inside a single class or a single method without too many abstractions. On the other hand, we didn’t want to make any changes on the monoliths later, as we couldn’t exactly know what consequences a simple change would have. 

One thing I wish I had done from the start was bookmark those important classes and methods so I could find them faster in the future. I am sure I spent too much time locating the controller method, which handled the HTTP request, and following the method calls until I got to the actual business logic I was interested in. 

Still, we started understanding the main functionalities of each service and how to build and deploy them so we could get involved in the day-to-day operations and start troubleshooting real issues. It was crucial to see how the current owners handled those situations, as most issues have similar causes and need similar steps to fix them.  

For example, when a job failed, the procedure was to change its status to “pending”, and the service would retry that job. Another common issue was errors when storing some Unicode characters in the database that wasn’t configured to use the Unicode character set. Situations like those could fill a whole other article. 

The takeover 

Once the time came for us to take full ownership of the system, we felt confident we could handle it because we were already troubleshooting issues and even deployed a few bug fixes. With that sorted, our focus shifted to designing and developing the replacement system to migrate the customers and shut down the legacy one. 

Although this wasn’t a project with all the newest bells and whistles, I did learn quite a lot from working on it. The most important lesson was that most systems will become legacy at some point, and we must keep that in mind while designing and developing them. That way, the next group of engineers that comes along to maintain it won’t need to ask a million questions. 

The post From zero to full ownership of a legacy system in under 2 months  appeared first on ShiftMag.

Last year, I took the initiative to build a new internal developer portal with several other great engineers. We were all very enthusiastic, but there was one fundamental thing missing. Altogether, we had 0 to no experience with TypeScript and MERN stack, on top of which Spotify’s Backstage platform was built. 

All of us were passionate backend engineers with a few years of experience working in the infrastructure department. Our tech stack consisted of more traditional technologies like Java’s Spring and SQL databases. Naturally, the biggest concern was how we would handle technologies we weren’t familiar with and build user-friendly and intuitive interfaces. After all, we were backend developers who preferred writing command-line instructions, deprived of a sense of good user experience. 

 On the other hand, we had an ace up our sleeves – a deep understanding of how our platform works. 

 Contrary to popular belief, we pulled off deploying Backstage in our platform and integrating it with the existing tools without any major challenges. The biggest issue was (and still is) its adoption. 

So, where is the catch?

Backstage has its set of core features, but it is also possible to extend it with your own or 3rd party plugins.  Core features or 3rd party plugins usually work without much hassle – your custom configuration gets injected in premade modules through YAML files.  

You can build interactive forms with multiple steps without writing any React code. This core feature is called Software Templates. We wanted to facilitate the bootstrapping of the Redis cluster. We had the define form which accepts configuration and actions that will be invoked on submission. The form was again defined through YAML. We had to write those simple actions in TypeScript, but after all, which developer doesn’t know how to write a function in any language? 

  When we decided to improve Search, another core feature, the biggest effort was to optimize the PostgreSQL search engine and decide if it was worth going a step further and experimenting with Elasticsearch – in the end, it was.   

While setting up authentication and SSO, the challenge was to explore all existing methods used throughout our company and unify them under one. Again, a task better suited for a platform engineer. 

With custom-made plugins, there was a little bit more React / TypeScript work. You have to figure out React fundamentals and start writing code. Backstage already provides out-of-the-box React components that follow their design principles, so you won’t have to think about colors and paddings. If those components are not enough, you can use MUI components from which the Backstage components are derived. Don’t worry, the internet is brimming with code examples. 

In the end, our knowledge of the platform and underlying infrastructure played a major role in our experience with developer portal integration. Lack of experience with frontend frameworks surely slowed us down a bit, but we learned along the way. 

The struggle

The biggest challenge, however, was yet ahead of us. It was (and still is) transitioning users from the old toolset they’ve been accustomed to. Despite the significant improvements in user experience and the addition of new functionalities, most users remain hesitant. 

To illustrate, we have a 10-year-old application management dashboard that is extensively used, but also notorious for its bad user experience. We modeled the new one after it, with a better user experience in place, but our developers still prefer the old one. When we asked them why they were not switching to the new one, their answer was simply – we don’t trust it. 

We also developed several nice and shiny plugins, a few features based on user requests, and resolved bugs we introduced on the way. Despite these efforts, our Backstage has yet to catch momentum. 

We acquired most users when we organized a two-day hackathon, where each of the 9 teams built a plugin they needed. Only one plugin is extensively used, but it brought around 40 daily active users, which is only 5% of our engineering organization.  

Despite the struggle, I believe that with good marketing, workshops, and constant improvements, we will bring most of the engineers to use and contribute to Backstage. When that happens, I’ll make sure to let you know how we managed to do it. 

The post You don’t need frontend developers for Backstage integration. But you do need adopters. appeared first on ShiftMag.

The task at hand was to migrate from the AWS relational PostgreSQL database to our on-prem self-hosted solution. We assembled a mission-based team comprising a Developer, DevOps, and two DBAs.

The main driver of this migration was the steadily increasing monthly bill we got from AWS. If you don’t read all the fine print when navigating through AWS UI, you could be unpleasantly surprised by the amount of dollars you end up spending.

But, we chose AWS for a valid reason. We had a POC (Proof Of Concept) and wanted to quickly gather customer feedback.

We chose PostgreSQL at that time and opted for an “enhanced” Aurora version. Although we had an on-premise (in-house) solution, it was never battle-tested.

From there, our journey began…

Choose your technologies carefully

Our cluster was AWS Aurora PostgreSQL RDS with one writer and 4 standby servers. The servers were db.r5.8xlarge instances with 32 vCPU and 256 GB RAM with Aurora storage.

The storage solution provides almost instant (approx. 20 ms) replication, with 4 out of 6 commit quorum.

All servers use the same underlying storage, so commits from writers are propagated instantly to standbys, and asynchronous replication updates only the data in memory (shared buffers).

Reference from AWS PostgreSQL In-Depth Training.

So we had help from the beginning…

Replication was instant, so we hooked up the whole UI on our standby servers and got an immediate real-time response. Our solution was built on the assumption that changes from the writer server will be, nearly instantly, available to our standbys.

The writer server was offloaded by efficiently using the standbys. We used Enhanced monitoring, set up Cloud Watch dashboards, analyzed and tracked every query on Performance Insight, and created alarms for every anomaly spotted in the setup.

The level of observability and monitoring let us diagnose issues very quickly and correctly. Little did we know that that behavior differed widely from Vanilla Postgres. But we would find out soon enough.

Think about your data early

Our database DDL was structured to support the product launch and was never meant to be a long-term solution.

Data was added but never deleted. Tables were not partitioned, and indexes have metastasized with various workloads. This resulted in tables with terabytes and indexes with hundreds of gigabytes of size.

We had to restructure. Keep the traffic data together and distinguish it from read-only and configuration data.

Archiving was a very long and IO-intensive feature to develop, and it clashed with the migration efforts. During the product lifecycle, we had stale and test data we wanted to get rid of. We started data cleanup and removed gigabytes of data.

Again conflicting with the migration process.

AWS charges everything

We didn’t start with a huge cluster, but with time and feature development, user adoption, and traffic increase, we had proportional demands on our underlying DB hardware.

Fast forward to being approx. 4-5 years on the market, in production, we had huge I/O usages of our standbys. UI was sucking in two standbys, and their usage was through the roof in peak traffic hours, while the other standbys were battling the remaining read queries.

We also hooked up some analytic reports on one of the servers to provide better statistics for our users.

The monthly bill was rising steadily. Although we neglected costs apart from instance uptime, every I/O request had its price and so did our setup.

On-premise setup

Although our cluster was not tested with the same traffic AWS was handling at the moment, our setup was powerful.

We had complete data center redundancy, meaning if a whole DC crashes, we could failover to the other one. Each DC had servers spread across three availability zones, so redundancy on the network level was also covered. Cluster orchestration with Patroni allowed for smooth failovers/switchovers and performing upgrades without downtime or maintenance window scheduling.

Hardware-wise, we had Intel’s Platinum CPUs, a few generations better than we had provisioned on AWS. Storage was NVMe SSDs, so IO bottlenecks we had on AWS should be history.

Use your standbys with caution

PostgreSQL allows you to use standby servers in a variety of ways:

• High availability where the standbys’ primary purpose is not to fall back from the writer and to replay WAL logs
• Offloading big queries, AKA poor men’s data warehousing
• Horizontal scaling, performing select queries to off-load the writer database

Every setup comes with its pros and cons, leading to streaming replication conflicts. We needed to choose whether we preferred replication changes or serving the current queries running on the standby(s).

While the abilities of configuration are broad, they are not bulletproof. The development mindset had to change, since we were now dealing with streaming replication instead of the one provided by Aurora (which was almost completely synchronous without latency increase).

Applications using standbys need to cope with eventual consistency. UI architecture for displaying real-time data from standbys in the new setup became an oxymoron from a logical perspective.

Limitations of logical replication

Logical replication was used to transfer data from the current cluster to AWS. We had previous experience migrating data but on a completely different DDL layout (remember the portion of the text “Think about your data early“).

The replication process has two phases. Initial data sync, where all the data currently in the table/schema is migrated, and the “catch-up” phase where the cluster syncs the subsequent changes from WAL logs.

Since logical replication does not copy DDL changes to the target server, all changes were frozen for other developers working on the product.

Our efforts were primarily aimed at reducing the cost, but we’ve managed to give ourselves some extra work in order to bring more value and earnings to the product.

The migration started by introducing replication slots on the source (AWS cluster) and subscribers on the target (on-premise). The first strategy, empirically, was to define one replication slot per database schema, as we did in our previous migration(s). Smaller schemas, up to 50 GB in size, were synced extremely fast and we experienced no issues.

Incidents from database degradations

By the time we got to our main schema counting several TB of size, the first incident happened. During the initial sync phase, WAL logs with subsequent changes are kept on the primary server. Given the way PostgreSQL provides transaction isolation, initial sync is thought of as a long-running transaction on the primary instance.

The snapshot xmin value is held and with it the event horizon. When this value is locked all queries “see” the DB state from the moment the transaction with xmin value started. In layman’s terms, every query has to start the execution from the snapshot of the DB that it was in at the moment of the ongoing transaction and go through changes (tuple versions) applied in the meantime.

This results in prolonged query cost, inaccurate database statistics, and the most problematic one bloating of the database since the vacuum is postponed. Approximately 5-6 hours in the replication we experienced complete degradation. IOPS got exhausted and we hit AWS limits, for our database instance level. We had to stop the replication, lose all the data currently migrated, and start over.

On the second attempt, armed with the experience we had from the first execution, we first divided our replication slots inside the schemas. Tables small in size were merged into a single replication slot, and every other table got its dedicated replication slot. Additional monitoring and alarms were set up to check the bloat and queries that were first impacted in the previous execution. Bigger tables were stripped from their indexes, and we decided to add them after the complete sync was done. The only index kept was the primary key one, since logical replication needs at least one unique constraint upon applying subsequent changes from WAL logs.

The migration was still a struggle.

The whole team was checking dozens of metrics and also browsing through our UI solution examining delays upon usage. We had to stop the migration and start over more than once, but this time it affected only a single, problematic table. Once we got to the biggest table inside our database, we needed to remove the primary key as well and recreate it once the initial sync was replicated.

We managed to copy all of the data to our on-premise cluster and were in the sync phase. Features such as data removal were halted until we fully migrated and we were now maintaining two separate clusters.

Load tests with business-specific scenarios

Data was replicated, clusters were up and running, DDL was synced, applications were working without issues, and the overhead of logically replicating changes didn’t affect the AWS cluster. Now what?

We needed to put our HW to the test, and we needed a fallback if we ran into problems.

This is the moment where we regretted not putting that extra time into having a load test. Not any kind of load test – the one that specifically mimics our production behavior, a user on our solution, and the ability to multiply it by a custom number.

Luckily, the majority of our load was SELECTs. We used our load balancers to include standby servers from the on-prem solution together with production ones. This allowed us to easily fall back, and kick out of the balancer, in case of degradation.

The major downside of our approach was that every latency introduced in the system was immediately affecting the client. We have repeatable tests that monitor the solution and, in 99% of the cases, spot the degradation before the user does, but there is still that 1%. Apart from “testing in production”, another major downside is that there were too many unknowns inside the equation to have exact conclusions. There is always a probability something else is affecting the results when you are part of a live system.

During these tests, we saw that our backups were disturbing our workloads and that we would need to introduce a separate server for full and incremental backups, something we didn’t even think about while we were on AWS.

Amzon’s Performance Insight is worth every penny. Only when we stopped using it and lost the ability to check the lifecycle of every executed query and pinpoint the exact wait event causing the delay did we realize how powerful it was.

We had to implement our solution, and we did, but again, we learned that we took some things for granted when using AWS.

The patient recovered in no time

The final maintenance was scheduled for the complete migration of all services to the new on-prem solution. Here our preparation was on point. We’ve completely automated the process, we just needed to execute a few CLI commands to gracefully migrate the whole solution. Given the usage of our product, the only window we had, with minimal traffic and client impact, was from 0:00-3:00 UTC. Since we are in CET this meant a long night for us.

We were struggling not to fall asleep before the maintenance started and would probably struggle to sleep afterward. The migration went surgically precise, and our patient recovered in no time. The window was scheduled for a full three hours, and we managed to redeploy, clean up error logs, and validate in a little more than an hour.

Although falling asleep was hard because of the adrenaline spikes, the sense of accomplishment helped us rest our tired eyes.

Life after migration

Looking back at the journey of the migration, a pragmatic observer could spot the good and the bad of each setup we had. Product maturity, access to powerful on-prem hardware, gathered know-how inside the company and cost reduction made the decision of the migration a no-brainer.

However, there is a lot of work now on our hands that we just “threw money at” when using the cloud. Maybe the best thing to understand as an organization and as a developer is to recognize when you are ready to make this step.

Identify a step in your product lifecycle where such moves have to be taken. With increased usage of your solution, you should think about scalability and cost-efficiency not only on service level or feature bottlenecks but the ecosystem as a whole.

Definitely choose your technologies wisely in the beginning. When you introduce new tech (platform change,  database, framework, or even a library), build up expertise, check the knowledge inside the company, understand and take responsibility for bugs in the tech now being your bugs.

Don’t forget to think about scale. Premature optimization is the root of all evil, they say, but nevertheless, your design has to have a vision.

These decisions are not easy, but neither is our job. We are fortunate to grow as individuals, learning by doing and owning our decisions, making us better developers but also human beings.

The post Database migration: Developers’ open-heart surgery appeared first on ShiftMag.

We are a small AI startup, and we’ve learned that it’s (relatively) easy to do pilot projects when you are small, as you can get away with your solutions not being on an enterprise level. But once you have to go to production, it’s a whole new ballgame. That’s how we got to create FastStream.

It all started with our work on pilot projects with several large enterprise clients. We quickly realized that the key to a successful transition to production was seamless integration between our predictive models and existing streaming platforms.

However, we faced a challenge.

Java is the dominant language in the streaming ecosystem, while Python is the king of AI. This made it difficult to bridge the gap between the two worlds.

We looked for an existing tool but couldn’t find anything that fit our needs besides Faust, which is deprecated and not something we can rely on, not when working with large companies.

That’s when we decided to create FastStream. We wanted to develop a framework that would make it easy to develop scalable and robust streaming microservices in Python, with the added benefit of automated code generation and predictive models.

From FastKafka to FastStream

We started by calling it FastKafka, thinking we’d only use it internally and just for Kafka, but then decided to release it as an open-source, purely as a tool that might help someone else. Well, soon, we realized that it needed to be more than just a Kafka wrapper. 

The response of the community took us by surprise. We quickly crossed 500 stars on GitHub, but more importantly, we received valuable input in terms of feature requests, suggestions, and feedback from seasoned enterprise developers.

One of our key collaborators was Nikita Pastukhov, the author of the Propan framework. Nikita contributed his expertise to merge the best features of Propan and FastKafka into a single, unified framework — FastStream.

This collaborative approach helped us to create a robust and fully extendable API that supports not only Kafka but also other key streaming protocols.

Generate an application from a short textual description

FastStream is not just about simplifying the codebase. Its architecture supports automatic code generation because we designed it with speed, scalability, and maintainability in mind. Namely, code generation is, in terms of time, a small part of the development; much more time is spent on the specification, creation of documentation, and testing. 

In our case, we have automated the whole process, whose result is a specified, documented, and tested microservice. In other words, our first prototype can generate an entire application — complete with unit and integration tests — from just a short textual description of the requirements but at the level of quality expected in large companies.

It is absolutely mindblowing that a task that would normally take several hours, sometimes even days, for a single developer now takes about one minute.

All this also enables seamless integration of our predictive models and future-proof microservices by allowing real-time data predictions.

The post We built a Python framework bridging the gap between AI and streaming platforms appeared first on ShiftMag.

Toma Puljak is a Software Developer with over 7 years of industry experience. He is fluent in Typescript, and with a strong background in C#, C++ and Python. Puljak joined Daytona as one of the core team members and he is included in almost all of the development processes in the company.

Collaborative apps like Figma, Notion, and Google Docs foster collaboration and productivity by allowing users to interact with one another in real time but are challenging to develop. The biggest challenge is resolving conflict, and maintaining latency-free experience as more and more users join the app.

Toma Puljak, a developer and speaker at the upcoming City JS Belgrade conference, shared his experience with building collaborative apps with us. Toma has been working on developing IDE editor tool for almost 5 years, and has recently joined Daytona, a SDE (Standardized Development Environment) tool as a founding engineer.

Puljak is also a passionate speaker and loves to share his knowledge with the developer community.

What does it mean for an app to be collaborative?

Toma: A collaborative app is one that allows users to interact with one another in real-time and the use of collaboration features depends on the app itself. For example, users could write in the same document (think Google Docs), follow each other’s cursors through the app, or create different elements on a drawing board.

Collaboration should not be a distraction

Could you provide an example of a well-known collaborative app and discuss what aspects of its design and implementation make it successful?

Toma: Some of the most popular collaborative apps (amongst developers) include Google Docs, Figma, and Notion. I’d say that what makes their collaboration features successful is the ease of use. They are collaborative by design, you don’t need to enable anything and users can just join and start working together. Collaboration should not be a distraction but a seamless way to work together and enhance your productivity and the apps that I mentioned do that very well.

What are the benefits of collaborative apps both for users and businesses?

Toma: The benefits of having collaboration features for users are mostly centered around boosting productivity.

Collaborative apps enable users to get their jobs done quicker and more efficiently because they eliminate the back and forth communication between colleagues. For businesses, anything that boosts productivity of their users is always a plus but collaboration can be that extra “Wow factor” that might boost engagement and customer satisfaction.

Tell us more about the collaborative features one app can have.

Toma: I’ll start with the collaborative features that Daytona’s browser IDE has:

• Collaborative text documents: Users can collaborate on the same text document in real-time and see what each user is writing and where their cursor is located.
• Collaborative terminals: Users can work together in the same terminal process and see the same output
• Chat: There is a dedicated chat window where users can write messages without leaving the IDE
• Screen sharing: Users can share their screen directly in the IDE which is an awesome feature to have when users need to collaborate on more than what the IDE offers.

What other collaborative features apps could implement?

Toma: Yes! There are collaborative drawing boards (similar to Figma, users can create different drawings or elements to create designs, charts, etc.), voice and video chats, and tracking cursors. This feature allows users to see exactly where other users are moving their cursor which gives them context on what their colleagues are focused on.

The awesome thing about these features is that they can be expanded upon to include anything that their app offers to boost productivity and give that “Wow factor”.

Challenges: Resolving conflicts and scaling collaboration

What challenges do developers face when they want to make apps more collaborative? 

Toma: One of the biggest challenges when implementing collaboration features is how to resolve conflicts. This is very important when working on collaborative text because users can edit, for example, the same line of text, and the collaboration implementation is responsible for resolving any conflicts that may arise. That’s why I mostly choose to work with Yjs when implementing collaboration features. Yjs is based on Conflict-free replicated data types which automatically resolve any conflicts and ensure that users end up in the same state.

Another challenge might be scaling collaboration features. As more users connect to the same collaboration session, it’s important to maintain a latency-free experience and that’s something that a lot of implementations struggle with. That’s why it’s important to choose the right communication protocol.

For example, WebSockets or peer-to-peer communication. Yjs is here another life-saver because it comes with support for different protocols that developers can choose to ensure a smooth experience no matter how many users are working together.

How can you ensure that the collaborative features of the app enhance the overall user experience rather than making it more complex?

Toma: The most important thing to think about when implementing collaboration features is to make it as seamless as possible. Don’t create hurdles, just allow users to jump into collaboration sessions and start working together without worrying about any of the setup.

This could be achieved by sharing links or room identifiers or so on. Just make sure that the collaboration does not get in the way of the users’ regular workflow but enhances it.

The post Here’s what to have in mind if you want to develop a collaborative app like Figma, Notion, or Canva appeared first on ShiftMag.

API Bakery is a project generator SaaS. More specifically, it generates API services that can be used for web or mobile app backends or exposed to the public. In a way, it’s an API for generating APIs – with a twist. Based on a recipe that describes the project and data on a high level, API Bakery generates the complete ready-to-use project in a way that makes it easy to modify and build upon.

Code generation pros and cons

Generative AI helping programmers be more productive is all the rage now, but using programs to generate new programs is nothing new. As an industry, we’ve been doing that for decades. This is especially true for APIs. In the broadest strokes, an API can be understood as a specification of how to call (interface with) some program or service. And we programmers are lazy: if we have the spec, why not generate code for it?

The problem is two-fold: we can rarely automagically create all the functionality that we need; and auto-generated code is usually an ugly, unmaintainable mess.

The many code-generator tools that exist can’t handle all imaginable cases and situations. Because of this, they focus on covering the most common functionality, or what’s the easiest to automate. For APIs, that’s usually the client SDKs that wrap the HTTP calls and parse the responses. This works, but the resulting code isn’t as nice to use as something crafted by hand. On the server side, the generated code usually consists of empty wrapper methods that need to be filled in by the user to actually provide the working code.

This brings us to the second problem: having to update and maintain that code manually. Often, the generated code doesn’t conform to the coding style, best practices, or patterns of the framework or platform being used. Because it doesn’t have a wider context of the entire application, it can also be sub-optimal or have potential security issues. Nobody likes having to go in and deswamp that.

Trying existing code and API generators

On the other hand, code generators can save you a lot of time. In our web development agency work, we’ve worked on a lot of projects where the API was mostly standard CRUD (Create, Read, Update, Delete) methods, plus a few things like payments and project-specific flows. Writing this almost-boilerplate API seemed like a waste of time and clients’ money, so we started using existing project generators and building an internal tool to automate as much of it as possible.

Most modern web frameworks have one (or a few) project generators. We work with Django (a web framework for Python) a lot, and there are at least a dozen Django project generators, with the most popular being cookiecutter. However, we found that those generators can only provide a barebones skeleton for a new empty project. All those CRUD APIs we still had to write ourselves.

The more we tried to customize the generators to help us generate more code, the more we had to maintain those customizations, in effect creating our own framework on top of Django. This was definitely not the way we wanted to go.

We also tried using Swagger (an open API specification that can be used to generate documentation or client or server code) and found out that writing a complete OpenAPI spec was almost the same amount of work as writing the Django code for it!

How we built API Bakery

What we ended up doing was writing our own code generator from scratch, focusing on two things: making it take a description of a project only to generate what’s needed and making sure the generated code looks and works great, like it was written by an actual person on the team.

This means, for example, that the code passes code-style and quality checks (linters), has unit tests (we aim for 90%+ coverage out of the box), and that when you open any file in the editor, the layout, variable naming, etc. all feels “normal”. On the flip side, we wanted to make sure we didn’t generate any generic code that would do more than asked: we aimed for the simplest, straightforward solution like a human programmer would.

We sorely underestimated how hard it would be! It took us a few rewrites of the generator itself to make it to a point where we’re satisfied with the output and with the complexity of the generator.

Realizing how useful it could be to other developers beyond our agency, we decided to spin it off into a separate standalone software as a service – API Bakery.

How API Bakery works

API Bakery combines project scaffolding (creating a structure for a new empty project), database model generation, CRUD API generation, and tests (and other features, but these are the main areas). Of those, the project scaffolding part is straightforward and doesn’t need to be customized much – it’s the most like other generator tools like cookiecutter. For a new project, we just create a couple dozen files with (mostly) boilerplate code.

The database model generation needs to know the structure of your data. This structure is an input to API Bakery in a form of a JSON recipe. The recipe describes the database models and their relationships. For example, a webshop might have Products that are grouped into Categories. Each Product might have a few Variants (SKUs), and so on. API Bakery users configure this through a web-based UI, and it is passed as a JSON recipe to the project generator. Based on this, the actual code to create and manage the data is created.

The API generation builds on this. API Bakery generates the standard CRUD methods for accessing and managing the data through a REST API. The code for this is generated in such a way that it’s easy to add custom API (like payments or more involved data flows) later. We realize that no generator can generate all the functionality, so we take care to make it easy to expand it with custom code.

Finally, we generate automated tests. This has proven the most tricky part because tests must be tailored to the API for the custom data models. Things like default or required fields, allowed values, and permissions all must be handled here; there are tons of edge cases that we have to handle.

Finally, we have a Docker-based isolated environment in which we run the linters and tests to make sure the generated code does what it’s supposed to do. Automated tests with high coverage make this easier – we just run the tests we generated in the previous steps.

For the user, this all takes a few seconds, and the project is baked and ready!

The AI future?

Will AI replace us all? The jury is still out, but in the meantime, we’ve started exploring how it can make our projects better. For API Bakery, we’re very happy with the code generator and don’t think GPT or other coder AIs are there yet when it comes to creating a full non-trivial project. Where we see the value is in combining AI’s ability to understand imprecise, implicit, and context-dependent human instructions.

Our current integration takes advantage of this – users can describe their project in as much detail as they like in normal human language. We then use GPT4 to turn these instructions into a recipe, which then feeds into our existing code generator. The results have been striking. You can literally type one sentence and get a fully working and tested project for what you describe.

Where AI can be of great use is also in side-coding or co-piloting, i.e., helping the programmer write code faster and better. This is an area that API Bakery doesn’t currently cover – once you have the initial project generated, you’re on your own. Using AI to add custom code to an existing project and to go beyond CRUD API is an exciting proposition and within the realm of feasibility already.

With continuing AI progress, the life of API developers will just get easier!

The post API Bakery: Building APIs that build APIs appeared first on ShiftMag.

Occasionally, everyone needs to handle data stored in files. CSV files seem like the way to go and are often the first choice – but I’d rather call them an anti-pattern.

Let’s say a customer requests an Excel report containing specific data. If this is a one-time thing, we can conveniently send it via email. In most cases, however, the data needs to be sent regularly. Not all clients use sophisticated software like Kafka; even when they do, we don’t always have access to it. So we need a solution that is both universally available and easily understood. SFTP and CSV files seemingly fit the bill.

This solution seems optimal for sending data to an external client. The problem arises when an internal application needs to integrate with ours, requires some data, and we decide to solve it by exporting CSVs. At first glance, this seems a low-hanging fruit.

Everyone can download a file.

Everyone can read a CSV.

It’s already developed.

While it may appear that simple, ultimately, it proves to be more costly than using a dedicated component for data exchange. I learned my lesson the hard way. I’ve worked on both applications that sent and received data via files. Here are some of the issues I discovered along the way.

 Data storage

The first thing we need is a place to store the data. Since our data is stored in files, it must be placed within a file system. The decision we face is whether to store the files on the producer machine, consumer machine, or an intermediary machine. Each option has its advantages and disadvantages.

Using the producer machine results in the consumer bearing the entire networking load. The consumer will need to periodically retrieve the data through a PULL operation. In the event of a network link failure, the producer will remain unaffected. 

However, this solution has a drawback – when consumers are redundant, and there are multiple instances, each instance may attempt to download the same file. Consequently, the same data will be processed multiple times. To address this, each instance should reserve the file before initiating the download. This can be accomplished by appending a suffix to the file name or moving the file to another folder. Once the file is reserved, the consumer application can proceed with the file download.

Using the consumer machine puts all the networking burden on the producer. The producer needs to PUSH the data to the consumer machine once the file is created. The consumer machine can watch the local file system for changes and react when a new file is created. As a result, the push-based approach can yield a faster response by the consumer and does not put any networking burden on the consumer. Since the file is sent directly to the consumer instance, this approach does not need file reservation. 

Pushing files to consumers also has a considerable downside – the producer must balance these files between the consumer instances. Implementing effective client-side balancing is a tough nut to crack. The producer would need to adapt its balancing strategy when a consumer instance gets offline or online and is added to or removed from the group.

Using an intermediary machine combines elements of both previously mentioned approaches. It eliminates the need for client-side balancing, but the consumer still needs to reserve the file before downloading it. The data storage is independent of both the consumer and producer, although both parties are still required to transfer files over the network.

Data retention

Regardless of the chosen method for storing data, it is crucial to ensure that the disk is not filled with files. Failing to do so will render the machines (and the services running on them) unusable. The primary line of defense is to store files on a separate section of the disk. 

Even if the partition gets filled with files, it will not impact other services. This advice applies even when files are stored on an intermediary machine, as some operating system functionalities may not work properly if the disk space is occupied.

Accumulating files can happen quite easily. For instance, the consumer application may be down (or not deployed) while the producer application continues to generate files. It can also occur if the producer generates a high volume of data the consumer cannot keep up with. 

In such cases, it is necessary to prevent the files from overflowing the disk. If the data represents events and skipping a few events in the event of a failure is acceptable, a retention policy should be implemented. 

For example, a script that periodically deletes the oldest files can be developed. This script adds another component to the architecture that needs to be created, maintained, and monitored. If the script malfunctions or contains a bug, the data retention may not work as intended.

Data processing

A file is removed from the disk when a consumer has read and processed the whole file content. The ‘whole file content’ is emphasized since a file represents a batch of data. Errors or failures can happen at any point between parsing and processing the data. If you delete the file without processing all the data contained within it, the data will be lost. On the other hand, if you do not delete the file but have already processed some of the data, you will end up processing the same data twice. This poses a data integrity concern that needs to be addressed. Both discarding data and processing data twice are undesirable and potentially unacceptable. That is why it is necessary to process a file in its entirety before deleting it.

This can be challenging, especially if the application involves multiple processing steps, including several database writes. Errors occurring in any of these phases can result in the entire batch being reversed. Additionally, processing a batch of data within a single transaction restricts the data processing to only one thread. File reading and data processing should be performed in separate threads to optimize performance.

One possible solution is to relax the data integrity requirement by implementing a queue where all the data is stored after parsing. This approach considers a file processed once all its records are inserted into the queue. Records can then be dequeued one by one, preventing the entire batch from being reverted. The issue with this approach is that the data in the queue will be lost in the event of an abrupt application failure. Since abrupt application failures are expected to be rare, this trade-off may be acceptable. If preserving data is not acceptable, a persistent queue for the data can be introduced (but we opted out of using new components in favor of using simple CSV reading).

It is also important to set a reasonable limit to the number of records in a file. A large file might present a huge load of data at once. So huge that it might crash the data if it does not have the memory to read the whole file.

Data Sharing

Once everything is set up and all the monitoring points are implemented (excluded from this article), we can enjoy our manually implemented file-based data exchange solution… until a new consumer is needed for the same data. 

Fortunately, we already have this functionality (file download+reservation, data retention, data queuing, monitoring, etc.) supported, and we can easily re-use it in any new consumer. Hopefully, the code has been written well enough to be copied and pasted into a library. Otherwise, there will be significant code duplication. Let’s not forget to create documentation for the module so that the solution can be integrated effortlessly.

Once we move this code to a module, our file-based data exchange solution will become a reusable component that any application can utilize (provided it is written in the same programming language). The only thing left is instructing the producer to export the files to a new location. There might be some duplicated data between the two locations, but let’s hope we will not add many of these consumers to the system.

Data storage already handles this – and it does it better

These are only some of the issues that we encountered. There are more of them, some not even mentioned here, and others are yet to be discovered. To resolve them, we had to invest time in development, refactoring, and as a result, we now have a code base to maintain. 

But these problems were not unique to us. They were common problems that most data storage software handles out-of-the-box and does it better!

The selection of software depends on the nature of the data. In our case, the data were events, and the producer application performed some batch processing on it. A more suitable storage option than the one we have been using would be a messaging system:

· Producers and consumers would write and read from a topic or a queue.

· Data would be stored and replicated to another machine, also enabling Geo-redundancy.

· Retention would be managed by the platform, eliminating the need for manual scripts.

· Messaging systems have a commit feature to indicate that a record (or batch) has been processed.

· Some messaging systems support consumer groups, allowing concurrent processing out-of-the-box.

· Different consumers (applications) can be subscribed to the same topic.

· Applications can be written in the most popular programming languages.

· Monitoring tools for popular systems are already available.

· Popular systems are usually well-documented.

For other types of data, the solution might be a more suitable data store like Redis or Postgres. It all depends on what the data is about and how it can be presented.

Exchanging data via files should be considered an anti-pattern and only be used when other approaches cannot be implemented (like when integrating with a legacy system). Each time some data is exchanged using files, a red flag should pop up, and you should ask yourself – what is a better way to do this? There probably is.

The post From file frustration to streamlined data exchange: Rethinking the approach appeared first on ShiftMag.

Almost every developer works with web APIs today, and the time they spend working with APIs continues to grow year over year. 

No wonder Joyce Lin, Head of Developer Relations of Postman, decided that her talk at the Shift conference in Miami would be titled “APIs are eating the universe”. Postman – a platform for building and using APIs, has 20 million users. In their State of the API report, 51% of respondents said that more than half of their organization’s development effort is spent on APIs. And 89% say API investments would rise or remain steady in the coming year.

You don’t have to be a developer to feel the impact of APIs in our everyday lives. They’re all around us.

Joyce Lin, Postman

Even developers who haven’t traditionally worked with web APIs are starting to get involved as the world becomes more interconnected, for example with data integrations and third-party services.

APIs live and die by developer experience

Asked to provide some words of wisdom for both API providers and API consumers, Lin has only two words to say: developer experience.

API providers that offer a smooth onboarding process have a developer mindset, and frequently this corresponds to the API itself in terms of design and support. You may think you have the best API on the market, but if it’s poorly documented and unpredictable, that’s a disservice to your consumers. Create onboarding guides to help reduce the time to value for your consumers. Watch new developers read through the docs and make their first call, and ask new team members to record their first experiences while they have fresh perspectives.

When it comes to API consumers, Lin says that sometimes the simple criterion of “it just works” can be enough when evaluating different API options:

Of course, you’ll want to make sure that the API is affordable, scalable, and can grow with your business. But don’t discount the value of an API that just works when evaluating different choices for APIs.

In her talk at the Shift Miami conference, Joyce reflected on the popularity of API protocols and patterns over time but also talked about what’s next in API development and how she sees it evolving. At Shift Zadar Joyce will show how to reverse engineer a private web API.

The developer community is a collaborative space

As a Head of Developer Relations, Joyce’s job is to share her knowledge through speaking, blog posts, or videos with the developers’ community. She got into developer advocacy when she was trying to find a job as a software engineer after attending a coding boot camp. She had previously worked as a product manager: 

Even though I had never heard of this type of role, I got a job as a developer advocate, which allowed me to be technical and share what I learned with the community.

Something about the developer community that I admire is that a lot of people do this on their own time, even if they don’t officially work in developer relations. It is a very collaborative space where people build open-source tools and projects and share their wisdom freely. 

Reaching more developers on TikTok than on Twitter

The space is becoming more and more crowded, with tonnes of content being created almost daily. At the same time, the average attention span is getting shorter. Joyce is one of the rare tech content creators exploring TikTok as a channel to reach developers.

I like to watch cat and dog videos and don’t personally use TikTok to learn about technology, so I am still surprised when people watch my TikToks about anything that I come across in my daily tech career, from web scraping to continuous integration.

I am able to reach way more people on TikTok compared to other platforms like Twitter or YouTube.

Even better, people are not just listening to what I have to say; they’re leaving comments and sharing their own ideas. So I get to learn as much from my viewers as they learn from me.  

It’s an easy and quick way for me to see what’s on everyone’s minds.

Joyce Lin, Postman

Sharing your ideas like this increases the discourse in the developer community and also helps independent developers who may not have access to the diversity of thoughts and trends available on the broader interwebs.

The post On APIs eating the world and creating TikToks for developers with Joyce Lin of Postman appeared first on ShiftMag.