Generative artificial intelligence has gone, in just a few years, from a technical curiosity to an everyday tool in newsrooms, legal departments, development teams and customer support centres. At its core are large language models (LLMs): systems trained on massive amounts of text that predict, word by word, the most likely continuation of a sequence. From this seemingly simple mechanism emerges the ability to write, summarise, translate, code and reason over complex instructions. This article explains what LLMs really are, how to address them through prompting, and how to integrate them into the business without falling into their traps.
What a large language model is and how it works
An LLM is a Transformer-architecture neural network with billions of parameters, trained in two phases. In pre-training, the model learns the structure of language by predicting the next token across a huge, heterogeneous corpus. In fine-tuning (including reinforcement learning from human feedback, RLHF), it is taught to follow instructions, to be helpful and to avoid harmful responses. The result is a system that does not consult a database of facts but generates text based on learned statistical patterns.
It is worth understanding one fundamental limitation: the LLM does not "know" in the human sense, it predicts. That is why it can produce hallucinations, plausible but false statements, with complete apparent confidence. Grasping this is not an academic detail: it shapes how applications are designed, which tasks are delegated to the model and where human verification is essential.
The context window and tokens
LLMs process text in tokens, fragments that may be whole words or pieces of a word. The context window is the maximum number of tokens the model can hold "in view" during an interaction: it includes the instructions, the documents supplied and the response itself. Large windows allow entire contracts or codebases to be analysed without chunking them, but anything that falls outside the window is simply unknown to the model. Managing context well, deciding what to include and what to summarise, is one of the key skills when working with LLMs.
Effective prompting: the art of instructing the model
The prompt is the instruction given to the model, and its quality determines the quality of the result. These are the techniques with the greatest demonstrated impact:
- Assign a clear role and goal: stating who the model should be ("you are a legal reviewer") and what it should deliver reduces ambiguity.
- Few-shot prompting: including two or three examples of the desired output format guides the model far better than an abstract description.
- Chain-of-thought: asking the model to reason step by step before answering improves accuracy on tasks involving logic or calculation.
- Delimit the context: clearly separating the instructions from the data (with tags or quotation marks) prevents the model from confusing the two.
- Specify the output format: requiring a concrete structure (JSON, table, list) makes the response directly usable by other systems.
RAG: connecting the model to your knowledge
To overcome hallucinations and the absence of recent information, the most widespread technique is retrieval-augmented generation (RAG). Instead of relying solely on what the model memorised during training, the system first searches its own document base for the fragments relevant to the query, and then hands them to the LLM as context so it answers grounded in them. This yields traceable responses, anchored in verifiable documentation and updatable without retraining the model. RAG is today the reference pattern for corporate assistants that must answer questions about internal regulations, catalogues or proprietary records.
Enterprise applications with real value
Beyond the hype, these are the uses where LLMs deliver consistent returns: support assistants that draft reply suggestions from the knowledge base; code generation and review under the developer's supervision; summarising meetings and long documents; drafting first versions of content that a professional polishes; structured data extraction from free text; and translation adapted to the brand's tone. The common pattern in successful cases is the same: the model produces the draft, the person validates and decides.
Comparative table of LLM usage modes
| Mode | Customisation | Cost | When to use it |
|---|---|---|---|
| Direct prompting | Low | Very low | General tasks and prototypes |
| RAG | Medium | Low to medium | Answers over proprietary, updatable knowledge |
| Fine-tuning | High | High | Very specific, repetitive style or format |
| Own model | Total | Very high | Extreme control or privacy requirements |
Common mistakes when adopting generative AI
The most expensive mistake is treating the model's output as verified truth and publishing it without review, especially in legal, medical or financial contexts. The second is dumping personal or confidential data into public tools without contractual guarantees about how it will be handled. The third is expecting perfect consistency: given the same question, the model may give different answers, which makes it necessary to design validations. The fourth is measuring success by the dazzle of the demo instead of by real business metrics.
Generation parameters: temperature, tokens and more
Beyond the text of the prompt, models expose parameters that modulate their behaviour and are worth understanding. Temperature controls randomness: low values (close to 0) produce deterministic, conservative responses, ideal for data extraction or tasks where consistency is critical; high values favour creativity and variety, useful for drafting ideas or brainstorming. Top-p (nucleus sampling) limits the selection to the most probable words until a certain probability mass is accumulated, another lever for constraining dispersion. The output token limit prevents truncated or runaway responses and controls cost, since most services bill per token consumed, on both input and output.
Understanding these parameters is the difference between an unpredictable system and a reliable one. For an assistant that fills in forms or returns JSON, a temperature near zero and a strict output format are essential; for an advertising headline generator, a higher temperature provides the variety being sought.
Regulatory compliance and governance
The deployment of generative AI in Europe is shaped by the EU Artificial Intelligence Act (AI Act), which classifies systems by risk level and imposes transparency obligations: the user must know they are interacting with an AI, and synthetically generated content must be identifiable, for example through watermarks or labelling. When personal data is processed, the GDPR applies, with its principles of minimisation, legal basis and data-subject rights, including the right not to be subject to automated decisions with significant effects without human intervention. The ISO/IEC 42001 standard provides a management system for governing these systems with policies, risk assessment and audit. An internal acceptable-use policy for AI, defining what data may be entered into which tools and who validates the outputs, is today as necessary as a security policy.
Frequently asked questions
What is the difference between a traditional chatbot and an LLM-based one? The classic chatbot follows predefined decision trees and only responds to what was programmed. An LLM-based one generates free language and handles unforeseen queries, at the cost of a greater risk of erroneous responses, which is best constrained with RAG.
Is the data I enter used to train the model? It depends on the provider and the plan contracted. Enterprise offerings usually guarantee by contract that data is not used for training. Verifying this before processing sensitive information is essential.
Is it better to train your own model? Almost never to begin with. Prompting and RAG cover the vast majority of cases at a fraction of the cost and risk. A proprietary model is only justified by extreme control or privacy requirements.
How are hallucinations controlled? By combining RAG to anchor responses in documents, asking for verifiable citations, lowering the "temperature" to reduce creativity, and keeping human review over critical decisions.
What role do AI agents play? An agent combines an LLM with external tools (search, calculator, system access) and the ability to execute several steps to solve a complex task. They extend what a model can do, but they widen the risk surface, so they require clear limits on which actions they can carry out without human approval.
Conclusion
Generative AI is neither magic nor an imminent threat to professional judgement: it is an exceptional productivity tool when its statistical nature is understood and workflows are designed accordingly. The organisations that extract real value are not the ones chasing the most spectacular demo, but the ones that scope concrete tasks, anchor the model in their own knowledge through RAG, keep a person validating what matters, and scrupulously comply with the AI Act and the GDPR from day one. Prompting is a skill that is learned; governance is a discipline that is built. At Summum Marketing we support companies on both fronts, so that generative AI becomes a reliable accelerator rather than a source of incidents.