Enterprise LLM Integration: Bridging Java and AI in Business Applications

An airhacks.fm conversation with Burr Sutter (@burrsutter) about:
discussion about integrating LLMs into enterprise Java
applications, challenges with non-deterministic LLM outputs in
deterministic code environments, limitations of chat interfaces for
power users in enterprise settings, preference for form-based
applications with prompts running behind the scenes, using LLMs to
understand unstructured data while providing structured interfaces,
maintaining existing CRUD systems while using LLMs for unstructured
data like emails and support tickets, practical examples of using
LLMs to generate code from business requirements, creating
assistants with system messages and short user prompts, potential
for embeddings to replace text prompts in the future, developer
journey in learning LLM integration including prompts, tools, RAG,
and agentic workflows, benefits of specialized agents over one
general agent, using LLMs for code generation with limitations for
complex use cases, hybrid approaches combining LLMs with human
oversight, using LLMs for email routing and support case
classification, potential for extracting knowledge from enterprise
data sources like Confluence and SharePoint, quality assurance with
LLM judges, discussion of small language models versus large ones,
model distillation and fine-tuning for specific enterprise use
cases, cost considerations for model training versus using
off-the-shelf models with better tool invocation, prediction that
models will become more efficient and run on commodity hardware in
the future, focus on post-training inference and reliable results

Burr Sutter on twitter: @burrsutter