Both approaches tend to be pretty vague. So it’s pretty rare that representatives of different telecoms—like Orange, Telefónica, Deutsche Telekom—share a stage and discuss not only the actual use cases of network APIs but also their limitations and the work that needs to be done to make them useful for developers.  

Cedric Gonin (Orange), Fernando Guillot Gimeno (Telefónica Innovación Digital), and Noel Wirzius (Deutsche Telekom AG) got pretty real(istic) onstage at Infobip Shift Conference, where they were joined by  Nokia’s Shkumbin Hamiti and Infobip’s Matija Razem.   

What are the use cases for Network APIs?  

The first issue on the table was – what do network APIs actually provide?  

Gonin explained that network APIs give developers access to certain user information that telcos use to identify their customers, like ISDN, MSIDN, SIM status, and other user information. This information is what you need to confirm that the device in question is really the one requesting information without sending information via SMS.  

It adds an additional layer of security against fraud and regular usage—after all, users can and do change numbers.  

Gimeno stressed that dynamic data is especially interesting to developers. Take, for example, API Device location—it has two uses, he explains. One is verification, verifying info that you already know—if you’re in front of the ATM, the bank can see if you are really around the ATM and the one getting the money. If you’re away, it can stop the transaction automatically.  

The other use is geofencing, meaning developers can set an area where they want an activity to occur. That way, a shop can offer a customer something when they are in the vicinity, or you could get a notification when your kids arrive where they are going.  

Of course, some of this can be achieved through a GPS signal, but GPS can be tampered with when it’s not coming from multiple antennas. Wirzius notes that getting the location from your telco without using your phone’s GPS is not only more accurate but also saves batteries. So it’s not just about inventing new use cases with Network APIs—it’s also about upgrading old ones using different technology.  

Use cases are great, but ease of use should be greater  

The what is important when it comes to Network APIs, everyone agrees, but the how holds as much weight. Quality of service is a part of mission-critical capabilities, and part of that is simplifying API usage and documentation.   

The telcos must embark on this journey together if they want NetworkAPIs to reach their potential. They noted that simplicity does not come naturally to them.   

After all, telcos rely on standards, as there is no other way to ensure interoperability. They have a natural tendency to introduce new standards when they face a problem, and that is somewhat at odds with their current mission of removing complexity from Network APIs.  

This is also where partnerships like the one with Infobip help by removing complexity for the end users – the developers. Let’s say you want to fly a drone over country borders. You cannot do this unless you can guarantee you’ll be able to operate it at all times, requiring you to have agreements with each telco that covers the area you want to fly it. Intermediaries like Infobip allow you to do this through a single API. They negotiate and make agreements with telcos all over Europe and worldwide so that developers don’t have to.   

In the words of Steve Balmer: Developers, developers, developers  

It’s also important to make things available to developers in a way they are used to, Hamiti stressed: “Linux foundation project Camara is one of the key efforts in that direction. Contributors help to scale it, but making it truly global requires close collaboration – between telcos and developers, but also amongst telcos themselves.” 

The first steps for developers, Gonin noted, would be to bring new ideas:  

“We are trying to do the same as with the first smartphones. We provide support for new ideas and create a platform for them. We can help bring new use cases to market and make them successful. We need to provide a full range to end-users to make Network APIs happen.”

Gimeno agrees:  

“The center point is developers. I worked for many years for Microsoft. I always remember Steve Balmer standing up in front of a huge crowd shouting, “Developers, developers , developers!”. Everything we are doing as integrators and telcos is making it easy for devs to do what they do.”  

He reminds us that it’s a new initiative—with 44 telcos participating, covering 22 countries. Yet if NetworkAPIs are global one day, telcos need to stop competing in their area and work together so developers don’t have to worry about what part of the country their users are in and what telco they are using.  

The post Network API: Achieving Simplicity on the Other Side of Complexity appeared first on ShiftMag.

Many things pass for AI, and sometimes it’s hard to put them under the common denominator, but Christine Spang still gives it a shot with a simple equation in her Shift Miami talk: Chatbots are cool, but what else can generative AI do?

Leverage = having a higher impact with a smaller input

AI = using computers to generate leverage

Computing itself, argues Christine, is about giving more leverage to individuals or groups, and the rise of LLMs has driven AI magic into new sets of use cases. 

We used to carry water to the village; now we have tap water. We invented language, and then systems for storing information. We keep making better ways to use the data and information we have today – AI is our latest attempt at that.

Chatbots, knowledge bases, coding assistants

Today, the most notable use cases are user-facing chatbots, knowledge bases, and coding assistants (or, as often happens, some combination of the three).

Chatbots have come a long way from their initial instances and can now boast great UI and conversational intelligence. 

Christine argues that coding assistance (or copilots) supercharges our coding powers, ensuring enhanced productivity and efficiency in the dev cycle. 

AI-powered knowledge bases give us access to the right information at the time when we need it, not when a customer service agent is available or can schedule a call.

As a good example of that (and the benefits that AI brings), Christine mentioned her company’s own chatbot, Nylas Assist, a chat user interface for their docs. Launched in August 2023, it has reduced the number of raised tickets by 25%, even though the user base grew by 30% – that’s precisely the leverage she’s talking about

Language is messy; data should not be

All three use cases are pretty useful – but is this the peak AI we’re experiencing? We’ve probably all guessed that it’s not. At the moment, Christine notes, we have generalized datasets, which only allow us to get generalized actions. 

Human language, on the other hand, is messy, full of nuances, and context-dependent. Communication is the bottleneck where most relevant information and context pass through, she states, and communication happens over many different channels, like messaging apps, email, voice, and social media, as well asynchronously.

Current LLMs are trained on the entire scrapable content available online. That’s a lot of text but not necessarily a lot of (right) context. LLMS trained on social media, Christine exemplifies, don’t necessarily have the right context for business. 

Customization is the next step forward

Communication data is a haystack, she stresses, it’s not valuable as an unstructured pile.

Take, for example, email, which is Nylas’s bread–and–butter data. It is particularly messy, with plain text, images, links, and formatting. Passing that data to a model the right way is a challenge. Everything needs to be pre-processed in a specific way, extracting not just information but also information order. It needs to be structured to have value and the right context to move from generalized outputs. 

Context is exactly how Christine sees generative AI evolving and generating even more leverage: When we access the context of a dataset rather than just the dataset itself, we can customize models and get customized actions instead of generalized ones.

The post The Magic and Reality of AI: What can Generative AI actually do? appeared first on ShiftMag.

Is prompting enough? Emanuel Lacić asked this question on the stage of the Shift Conference in Miami as he explored the process of creating a Copilot for a UI-based chatbot builder.  

The chatbot builder in question, Answers Copilot, is a GenAI feature that enables end users to design a chatbot based on their natural language input. GenAI creates an outline of the design of how the chatbot should behave, automating the chatbot building process to a degree, and the end user then customizes it to meet their requirement.  

Starting with prompting

The initial process relied on prompting: Emanuel and his team described what the underlying code looked like, had Open AI generate the code blocks representing visual elements, and then plugged it in to have it rendered in the UI. Preferably with as few hallucinations (i.e., generated code that leads to an error when rendering), and as predictable output as possible.  

They tested different prompt engineering strategies with Microsoft’s API for GPT-3.5 Turbo. By testing different techniques ranging from zero-shot to few-shot prompting with domain-specific instructions, they managed to lower the percentage of hallucinations to 12.63% on average.  Accuracy was measured using HitRate – the number of times where the generated code blacked matched to a 100% of what was expected – which peaked at 2.13%.

Having created the Copilot using different prompting strategies, it was time to answer Emanuel’s titular question: Is prompting enough? The team decided to test the hypothesis that LLMs with context-specific data might yield a lower percentage of hallucinations and higher accuracy (i.e., by measuring the HitRate and turning to fine-tuning. 

Bigger is not always better 

As end users can task the Answers Copilot with creating a chatbot for a variety of use cases, the task of fine-tuning it required the team to know what input users might provide, as well as what is the desired output. Since real-world data was not available, GenAI was put to the task of synthetically creating some.  

The data was then used to fine-tune LLMs of various sizes: OpenAI GPT-3.5 Turbo (large), Mistral 7B Instruct (mid), LLaMa 3B (small), and Sheared LLaMa 1.3B (tiny). In addition to training the models with relevant data, the team used LoRA to fine-tune visual element generation.  

The fine-tuning process did yield the desired results: LLMs trained on relevant data had a significantly lower number of hallucinations, with 0.04% as the lowest achieved hallucination rate. The accuracy, on the other hand, also improved significantly, where the HitRate climbed up to 26.72%. 

Interestingly, Emanuel notes the best performing models were Sheared LlaMA (in terms of hallucinations) and Mistral 7b Instruct (when it came to HitRate): 

Sometimes you don’t need the largest, best performing LLM. But the only way to know which one performs best is to experiment – you can’t know beforehand. 

What’s next? 

There are always ways to polish Copilots, with user feedback being the logical next step. To that end, he showed the KTO method (Kahneman-Tversky Optimization): As it requires only a binary signal (desirable/undesirable outcome), the user feedback data is more abundant, cheaper, and faster to collect than data based on user preference between two different outputs, which is used in other popular methods like Reinforcement Learning. KTO is also a good choice when there is a marked imbalance between the number of desirable and undesirable examples. 

To take user feedback a step further, a multiarmed bandit algorithm can be used, as Emanuel demonstrated, to determine which of the LLMs produces the most favorable results while running in production and, consequently, which LLM to choose in an automatic way.  

You can find Emanuel’s slides here or find out more about his work on his personal website. 

The post Want to build a more accurate Copilot with fewer hallucinations? Move from prompting to fine-tuning. appeared first on ShiftMag.

There is something to unpack here, says Baruch Sadogursky as he takes the stage at the Miami Shift Conference. A conspiracy theory, even. 

And here it is:

“DevOps is a conspiracy by Ops people to make developers work harder.”

Jokes aside, Baruch urges us to look into who teaches us about DevOps. A quick Google search along the lines of “What is DevOps” reveals that the answer to this question can be found on the websites of AWS (an Ops platform), Atlassian (a DevOps company), Gitlab (Offering DevOps in the box), Synopsis ( Observability platform). Is it a coincidence that all sources that teach us about DevOps are (dev)ops or DevOps-adjacent companies?  

Why do they want to sell DevOps ideas to us so badly? Is it about the revenue that the industry generates? A DevOps salary is almost double that of a system administrator (with not that great of a difference in the job description); there are almost 20k open positions for DevOps engineers in us alone, and let’s not forget about the certification business behind it. 

One could easily make the connection between money and the push for DevOps – but only if we forget to go back in time to when DeOps first started. When it was first conceived and gained momentum, the money was just not there, Baruch notes. 

This looks familiar

If we look at the DevOps cycle, it seems pretty familiar – it is basically an agile development cycle, with an Ops cycle plastered on there, and suddenly it’s supposed to all be dev work! Even when we look at the main aspects of DevOps work, we hear Ops concepts: deploying code, going from code commit to running code in production, unplanned outages, and degraded services.

Baruch protests that none of them feature in the basic postulates of dev work: Developers do new features, refactor, and fix bugs; that’s the job. The code passes the tests; it works, and the job is done. What happens in production is beyond the scope of work, so there is no room for these ops concepts there, right?

Well, it’s not that simple. If we take a look at a software craftsman definition of done Baruch provided, the list is much longer:

The project is done once the dev understands what needs to be done; the code is simple, readable, and easy to deploy; non-functional requirements are met; no tech debt was created; tests pass & QA is happy with the code; team lead, PO and client are all satisfied. That’s a much longer list, and what stands out is quality.

Baruch leads with another difficult question here: What is quality, that we’re so concerned about?

Testing in production = caring about what happens in production

Quality is an ever-evolving concept. It’s certainly not what it was ten years ago, as Baruch illustrates with an example of his old bank website, which allowed you to check your balances and transactions but nothing more complex. It was nice to have but not critical, so when the website was out for 12 hours for update & maintenance to ensure quality of service, that was not a big deal.

That era is gone – we expect more of our software now. In terms of banking, we do it via mobile app and expect everything to work at all reasonable times, and to work quickly and efficiently. Code quality alone is not enough to cater to that.

Let’s just look at the size of the global datasphere – 175ZB. There is more and more data in more and more companies, which makes testing harder. Since we have so much data in production that we cannot, or at least it’s not worth replicating in staging. So, we test in production:

And if we’re testing in production, and being in production is what is needed to ensure the software quality that’s so important to us as developers, that means we now care what happens in production.

And if we care about what happens in production, that means we have to be there when we throw untested software on unsuspecting customers in production and observe the blast radius from our bunker. We have to be there in the middle of the night with the ops people because this is actually the first time we get to test our software.

The new scope of work

This new turn of events means that there are some more items on the developers’ to-do list:

• We understand how our software is going to be deployed
• The build is reliable, repeatable & fast
• The code is stateless & scalable
• It starts fast and dies fast
• It is observable
• It supports feature flags
• It’s backward & forward-compatible
• The code emits event streams

This is what you need to ensure the quality of your software in production, says Baruch, so starting now, you care about this stuff:

10 years in, developers are still not excited about DevOps. But still, we have to be there – because we care about what we’re doing. So how do we go from to :).

So, is DevOps a plot by Ops people? For sure, Baruch says, but it’s also an evolution and a means to an end. The end being quality, new features, lean software and security. DevOps just delivers what every business needs.

DevOps engineers are rainbow-farting unicorns

So now that you’re thrown into DevOps, you better get to know it. If we look at the traditional DevOps Venn diagram, we can see that DevOps is an intersection between Devs, QA, and Ops. So, are you supposed to master all those disciplines? If that thought terrifies you, Baruch has some good news: 

That person does not exist. It is a rainbow-farting unicorn, completely made up. DevOps is not about one person; it’s about collaboration. 

Instead, he suggests there are T-shaped people: People who know their job (coding) well but also understand what the Ops and QA part of the house means. This doesn’t mean developers have to become QA and Ops; they just take an interest in what they do. 

That’s not so bad, is it? Or, in Baruch’s own words: 

“DevOps is fine.”

The post Is DevOps just a conspiracy theory? appeared first on ShiftMag.