In 2025, 20% of enterprises in the European Union used artificial intelligence, compared with 13.5% a year earlier. In Poland, the figure was 8.4%. The most common application was analysing written language, used by 11.8% of the companies surveyed. The authors of a study published in 2026 in Organization Science described the uneven boundaries of AI capabilities as a “jagged technological frontier”: tasks of similar difficulty for humans can pose very different challenges for a model. The next step requires answering a more specific question: which form of AI fits a particular task?
In an experiment involving 758 consultants, participants using GPT-4 completed 12.2% more tasks and finished them 25.1% faster on average when the tasks fell within the model’s capabilities. For a complex task beyond those capabilities, the probability of reaching the correct solution fell by 19 percentage points. The authors of the study, published in 2026 in Organization Science, called this uneven boundary of AI capabilities the “jagged technological frontier”.
Effectiveness therefore depends on matching the technology to the task, data, risk and approach to verifying results. One process may be best served by an enterprise LLM assistant. Another may require an application connected to a CRM, a knowledge base and an access control system. A third may benefit from a small model running locally. Operations governed by explicit rules can be handled by code, rules engines or robotic process automation (RPA). Traditional machine learning is suitable for tasks such as forecasting and data classification.
1. LLMs and SLMs in Business: Choosing the Model, Integrations and Where Data Is Processed
Terms such as “LLM”, “deployed LLM” and “closed SLM” combine several layers of technology. In a business context, it helps to separate four decisions:
Way of working: does an employee interact with a ready-made assistant, or does the process start automatically?
Scope of integration: does the solution work with materials supplied by the user, or does it retrieve data and perform actions in company systems?
Model type: does the task require the broad capabilities of a large language model, or would a specialised SLM be sufficient?
Processing location: does the model run as a cloud service, in a dedicated environment, in a private cloud, on premises or directly on a device?
SLM stands for Small Language Model, which typically requires fewer computing resources. The term “closed system” needs clarification: it may refer to restricted access, an isolated environment or data processing within the organisation’s own infrastructure. A large language model can run in a private environment, while an SLM can be available through a public API. Model size alone does not determine how data is secured.
2. Six Ways to Use AI in Business Processes
Approach
How It Works
Best Fit
Key Metric
Enterprise LLM assistant
An employee assigns a task and checks the result in an approved environment, such as ChatGPT Business or Enterprise
Analysis, drafting, summarising, developing alternatives and ad hoc work
Time saved per task after accounting for review and corrections
Integrated LLM or RAG application
The model uses company sources, rules, permissions and integrations
Repeatable processes involving documents, knowledge and data from business systems
Cost per successfully resolved case
AI agent
The solution plans its next steps, selects tools and pursues a goal within a defined scope
Multi-step processes involving exceptions and a dynamic sequence of actions
Percentage of tasks completed correctly
SLM
A smaller model handles a narrow range of tasks in the cloud, on a company server, at the edge or on a device
High task volumes, a fixed subject area, short response times and offline operation
Quality compared with an LLM within a specified cost budget and p95 response time limit
Private or on-premises deployment
An LLM or SLM runs in a controlled environment, private cloud, company network or on a device
Requirements relating to data residency, business continuity, connectivity, infrastructure or security policies
Compliance with requirements, quality, availability and total cost of ownership (TCO)
Rules, RPA or traditional ML
The process is defined through code, conditions, a predictive model or a state machine
Calculations, transactions, fixed workflows and unambiguous decisions
Accuracy, repeatability and completeness of the audit trail
In a mature deployment, these approaches often work together. The language model interprets a message, rules check the conditions, an application retrieves data, a person approves the action, and the transactional system records the result.
3. Which Tasks Are Suitable for ChatGPT or Another LLM Assistant?
A ready-made LLM assistant supports tasks where an employee is responsible for checking and using the result. The user initiates the work, provides context, evaluates the response and decides how to use it. The output takes the form of a draft, recommendation, analysis or working document.
Examples include:
preparing a first draft of a report, message, presentation or article,
summarising documents and correspondence,
comparing several materials supplied by the user,
developing questions, scenarios and alternative solutions,
translating content for subsequent review,
exploring data and explaining findings,
organising meeting notes,
drafting a procedure or action plan.
This approach works well in processes where every response undergoes human review, the result can easily be corrected or withdrawn, and the task does not require automatic writes to a critical system. Wide variation in the source material and the need for language-related work further increase the usefulness of an LLM.
Security depends on the approved product and its configuration. OpenAI states that data from ChatGPT Business, ChatGPT Enterprise and the API is not used to train models by default. The API data controls documentation also describes separate retention policies, including standard abuse monitoring logs and a Zero Data Retention option for eligible customers. Before using the solution, an organisation should review data classification, contractual terms, processing region, retention, administrator permissions and policies for connected applications. For more examples, see our overview of 15 ChatGPT integrations with business applications.
3.1 How Can You Measure Time Savings and the Quality of Work with an LLM?
Relying solely on employee surveys can overstate the benefits. In a METR experiment, experienced developers took 19% longer to complete the tasks studied when using AI tools, yet afterwards still estimated that AI had made their work 20% faster. The study involved 16 participants and 246 real tasks in repositories they knew well, so its findings apply to that specific setting. The methodological lesson has broader relevance: actual task duration and quality need to be measured before deployment.
For an LLM assistant, useful metrics include median task completion time, the proportion of outputs accepted without changes, average correction time, quality assessed against consistent criteria, frequency of use and the number of cases in which users return to their previous way of working.
4. When Should You Integrate an LLM with Company Systems, and When Should You Deploy an AI Agent?
Integration becomes justified when the value of a process depends on current company data, a repeatable workflow and coordination across several systems. The model then receives controlled access to documents, a knowledge base, CRM, ERP, a ticketing system or email. The application verifies the user’s identity, controls which data is shared, defines the response format, checks results, logs actions and routes selected operations for approval.
A typical integrated AI application consists of five layers:
Input: a message, document, form, system event or record.
Context: data retrieved in line with access permissions, often using RAG.
Model: an LLM or SLM selected for the specific stage.
Validation: rules, completeness checks, source verification and risk classification.
Output: a response for a person, a draft record or an approved action in a system.
Use cases requiring this architecture include finding answers in an internal knowledge base, reviewing contracts against a company’s risk checklist, preparing quotations using CRM data and a price list, classifying support tickets, onboarding employees, checking procurement documents and drafting responses based on customer history. RAG retrieves up-to-date passages from approved sources and adds them to the context used to generate a response.
An AI agent represents a further level of integration. It receives a goal, selects tools and plans a sequence of actions. According to the current Google Cloud guidance on agentic architecture, agents are suited to open-ended, multi-step problems that require external data and a degree of autonomy. An application with a predefined sequence of steps is usually sufficient for a single summary or translation.
We explore the role of an advanced model as a reasoning layer connected to tools, data and permissions in our article GPT-5.5 for Business: A New Era of AI Agents.
The scope of an agent’s autonomous actions can be expanded when test results confirm the required effectiveness and safety. Its permissions should be limited to the functions needed for the process, read access should be separated from write access, and actions with significant consequences for the company or customer should require approval. OWASP identifies excessive functionality, excessive permissions and excessive autonomy as the three main causes of Excessive Agency risk. The principle of least privilege limits the consequences of misinterpretation, fabricated information or an attack that feeds malicious instructions to the model (prompt injection).
5. Which Business Processes Are Suitable for a Small Language Model (SLM)?
A Small Language Model uses fewer parameters and computing resources than a large language model. According to Microsoft Azure, this can support faster responses, lower infrastructure requirements and data processing close to where the data originates (edge computing), for example on industrial devices. A specialised SLM can be suitable for tasks with a fixed subject area, predictable inputs and a clearly defined output.
An SLM is worth testing when a process meets several of the following conditions:
a fixed set of categories, intents, fields or response types,
a high and predictable volume of requests,
a requirement for a short response time measured as p95, the threshold within which 95% of responses are completed,
limited hardware resources,
a need to operate offline or directly on a device,
access to data from the relevant business domain and a set of reference answers,
the ability to escalate difficult cases to a larger model or a person.
Examples include classifying support tickets into 30 fixed categories, identifying a user’s intent, extracting field values from a single document type, generating short responses within a tightly defined subject area or analysing messages locally on an industrial device.
The choice depends on testing with company data. An SLM should meet the required targets for quality, response time and cost per correctly handled case. For example, a company might require the smaller model to retain at least 98% of the reference LLM’s quality score, reduce the cost per correct result by at least 20% and stay within the p95 response time limit. These are illustrative decision thresholds that the process owner sets before the pilot.
5.1 When Should You Deploy an LLM or SLM On Premises or in a Private Cloud?
A private or on-premises deployment may be driven by requirements relating to data sovereignty, security policies, business continuity, network latency or operation without internet access. Such a deployment can use an SLM or a larger model. An enterprise application can also use a managed API with encryption, retention controls, an appropriate processing region and contractual provisions governing data handling.
A cascading architecture can be the most effective approach. Microsoft describes a hybrid model in which an SLM handles routine queries and passes more complex cases to an LLM. In a business setting, it is worth adding a third route to this cascade: referring the case to an employee when the result is uncertain, the risk is high or required data is missing.
6. Which Processes Should You Automate with Rules, RPA or Machine Learning?
Processes governed by fixed rules require a clearly defined sequence of actions and conditions for carrying them out. AWS documentation on orchestration distinguishes between rule-based workflows, where successive states and transitions are explicitly defined, and agentic orchestration, where a model interprets the goal and dynamically selects tools. Both layers can operate within a single application.
Process
Recommended Mechanism
Role of the Language Model
Calculating tax, pay or a discount
Code and a rules engine
Explaining the result or interpreting the user’s query
Executing a payment, refund or limit change
A transactional process with access controls
Identifying intent and preparing data for approval
Granting or revoking permissions
Identity and access management (IAM), role-based rules and approvals
Handling a request expressed in natural language
Checking that all required fields are complete
A schema validator
Extracting fields from an unstructured document
Predicting customer churn or forecasting demand
Traditional machine learning
Explaining contributing factors and drafting communications
Interpreting a free-form message
An LLM or SLM
Classifying intent and passing data to a controlled workflow
In a financial process, an LLM can read a message, identify the request and prepare a proposal. Rules check the balance, limits, customer status and required approvals. The transactional system executes the operation once the conditions are met. Each layer performs a task for which clear criteria for correctness can be defined.
7. How Do You Match AI to a Business Process? Four Assessment Criteria
An initial assessment can be carried out during a short workshop. The process owner evaluates the process across four areas, assigning a score from 0 to 3 in each. The individual scores help define requirements for the model, integrations, safeguards and infrastructure.
Dimension
0
1
2
3
Complexity of content interpretation
Fixed fields and rules
A fixed set of categories
Interpreting context
Combining information and reasoning across multiple sources
Integration and autonomy
No access to systems
Reading from a single source
Reading from multiple sources or preparing data to be written to a system
Transactions and dynamic tool selection
Impact of an error
Easily reversible
Limited operational cost
Significant financial, legal or reputational impact
Critical or irreversible consequences, or an impact on rights and safety
Infrastructure and data requirements
A managed cloud meets the requirements
A specific region or retention policy is required
A private network or strict latency limit
Operation offline, in an air-gapped environment, at the edge or directly on a device
The scores can be interpreted as follows:
Content interpretation complexity of 0-1 with stable rules: code, workflows, RPA or traditional ML.
Complexity of 2-3, integration of 0-1 and error impact of 0-1: an enterprise LLM assistant with user review.
Complexity of 2-3 and integration of 2-3: an integrated LLM application, RAG or an agent.
Complexity of 1-2, a narrow domain and infrastructure and data requirements of 2-3: an SLM as a candidate for comparative testing.
Error impact of 2-3: approval by an authorised person, safeguards based on predefined rules and a complete activity log, regardless of model type.
The table helps identify solutions for a pilot. The final decision follows a comparison of their performance on the same set of real cases.
8. ChatGPT, Integrated LLMs, SLMs and Automation: Business Use Cases
Example Process
Recommended Architecture
Key Performance Indicator
Human Oversight
Drafting marketing content
An enterprise LLM assistant
Median time saved and percentage of outputs accepted
Approval of every publication
Summarising a meeting and listing action items
An enterprise assistant with access to an approved source
Completeness of action items and number of corrections
Verification of task owners and deadlines
Answering questions about internal procedures
An integrated LLM with RAG and source citations
Percentage of answers grounded in sources and accuracy of citations
Escalation when no source is available
Assigning support tickets to predefined queues
An SLM or classifier, with an LLM for exceptions
Macro-F1 and the proportion of priority tickets correctly identified
Review of uncertain cases
Reviewing contracts against a company’s risk checklist
An integrated LLM with RAG, rules and logging
Rates of detected and missed risky clauses
Decision by a lawyer
Extracting fields from a single invoice type
OCR, traditional ML or an SLM, combined with rule-based validation
Field-level accuracy and cost per document processed
Review of exceptions
Preparing a quotation using CRM data and a price list
An integrated LLM, RAG and price retrieval governed by predefined rules
Preparation time and percentage of quotations requiring commercial corrections
Approval of pricing and terms
Checking refund eligibility
Process rules
Compliance with policy and time to decision
Handling exceptions
Executing a refund
A transactional workflow with authorisation
100% accounting accuracy and a complete audit trail
Depends on the amount and risk
Analysing machine messages locally
An SLM or specialised model running at the edge
p95 response time, alarm detection rate and availability without an internet connection
Escalation of critical alarms
Revoking access for a departing employee
IAM and a deterministic workflow
Completeness of access revocation
Approval in line with policy
Handling a customer case across multiple steps
An AI agent with a restricted set of tools
Task success rate, correctness of tool use and percentage of cases referred to an employee
Approval checkpoints for high-impact actions
9. How Do You Measure the Results of an AI Deployment? Quality, Time and Cost Metrics
Measurement starts with the existing process. The Generative AI at Work study, involving 5,179 customer support employees, found an average increase of 14% in the number of issues resolved per hour. Less experienced employees saw the greatest improvement. Productivity defined this way has a clear numerator, denominator and comparison group. An enterprise pilot requires similar precision.
Area
Metric
How to Measure It
Scale
Case volume
Number of cases per month, seasonality and peak demand periods
Time
Case handling time
Mean, median and p90 before and after deployment
Quality
Task success rate
Percentage of cases meeting all criteria for correct task completion
Usefulness
Percentage of outputs accepted without substantive changes
Proportion of outputs accepted without changes to their substance
Oversight
Percentage of AI decisions changed by an employee
Proportion of decisions or proposals modified by an employee
Risk
Critical error rate
Number of critical errors per 1,000 or 10,000 cases
Classification
Precision, recall and F1 score
Measured separately for each important class, particularly rare events
RAG
Consistency of responses with sources
Percentage of claims supported by a cited, up-to-date source
Agent
Correctness of tool selection and supplied parameters
Whether the correct tool was selected and valid parameters were supplied
Automation
Percentage of cases handled entirely automatically
Proportion of cases completed without manual intervention
Performance
Time from task initiation to the final result
p50 and p95 of end-to-end task completion time
Economics
Cost per successfully completed task
Total cost divided by the number of correct outcomes
Stability
Changes in system quality over time
Variation in quality by time period, language, category and user type
Google Cloud identifies cost per successfully completed task as a key metric for AI agents operating in real business processes. A model that costs USD 0.10 per run and achieves a 50% success rate incurs a model invocation cost alone of USD 0.20 per correct result. Human review, retries, integrations, monitoring and the cost of errors must also be included.
10. How Do You Calculate the ROI of an LLM or SLM Deployment?
The full cost of the solution should include development and integration, the model or API, infrastructure, monitoring, updates, human review, corrections and expected losses resulting from errors.
Cost per successfully completed task:
C_success = (development cost allocated to the period + model/API + infrastructure + monitoring + human review + corrections + expected losses from errors) / number of successfully completed tasks
Annual benefit:
Benefit = value of working time saved + avoided correction and error costs + additional margin + avoided SLA penalties
ROI:
ROI = (Benefit – TCO of the AI solution) / TCO of the AI solution × 100%
Suppose a team classifies 20,000 support tickets per month. Each ticket takes an average of 4 minutes, and the fully loaded hourly labour cost is PLN 120. The monthly cost of manual classification is approximately PLN 160,000. During the pilot, 88% of the system’s outputs are accepted without correction. Reviewing each output takes an average of 45 seconds, correcting the remaining cases takes 3 minutes each, and the monthly costs of the model, infrastructure, maintenance and amortised implementation total PLN 51,000.
reviewing all cases: approximately PLN 30,000,
correcting 12% of cases: approximately PLN 14,400,
model, infrastructure, maintenance and implementation: PLN 51,000,
total process cost after deployment: approximately PLN 95,400,
monthly cost reduction: approximately PLN 64,600, or 40.4%.
This example illustrates the calculation method using assumed values. A financial assessment of an actual deployment must account for changes in the cost of errors, seasonality, downtime, exception handling costs and the pace of employee adoption. For a revenue-generating process, the calculation should also include changes in margin, conversion rate or customer retention.
11. How Do You Run a Measurable LLM or SLM Pilot?
Establish a baseline. Measure volume, time, quality, errors, escalations and the cost of the current approach over at least one full business cycle.
Prepare a test dataset. Include typical tasks, difficult and rare situations, boundary cases and deliberate attempts to mislead the system. Google Cloud recommends a custom dataset that reflects the full range of intended uses.
Set thresholds before testing. Document the minimum quality, maximum cost, acceptable latency, critical error limit and escalation rules.
Compare several approaches. Include the current process, a capable LLM, a smaller model and a hybrid architecture.
Run the system in shadow mode. AI generates outputs alongside the existing process. Decisions and actions continue under the established rules. This helps identify errors before AI is allowed to handle live operations.
Deploy the system within a limited scope. Start with proposals and approvals. Expand automated actions based on the results.
Monitor performance quality. Repeat testing whenever the model, AI instructions, tools, data sources, rules or types of input material change.
11.1 How Many Test Cases Do You Need to Evaluate an LLM or SLM?
For a metric expressed as a proportion, a conservative sample size at a 95% confidence level and a margin of error of +/-5 percentage points is approximately 385 independent cases. A margin of +/-3 percentage points requires approximately 1,068 cases. The representative sample should be supplemented with a separate set of critical and edge cases.
For rare errors, the so-called rule of three is useful. If no critical errors occur in 300 tests, the approximate upper bound on their true probability at a 95% confidence level is still around 1%. Zero errors in 3,000 tests reduces that bound to approximately 0.1%. High-risk processes therefore require much larger datasets and tests targeting specific threats.
11.2 When Should You Complete an AI Pilot and Move into Production? Example Criteria
Process
Example Criteria for Moving into Production
Support ticket classification
Macro-F1 of at least 0.90; recall for priority tickets of at least 0.99; override rate no higher than 8%; p95 response time no longer than 2 seconds
Internal knowledge base
Acceptance rate of at least 85%; citation accuracy of at least 98%; no unsupported claims in the critical test set; p95 response time no longer than 8 seconds
Draft quotation
Median preparation time reduced by at least 30%; at least 75% of drafts accepted with minor changes; 100% of prices retrieved from an authorised source; human approval for every quotation
These values illustrate how to define acceptance criteria. The process owner sets the thresholds according to the cost of errors, required quality and the organisation’s risk tolerance.
12. How Do You Match Human Oversight to AI Risk?
NIST defines risk as a combination of the likelihood of an event and the scale of its consequences. This principle translates general concerns about AI into a measurable model: error frequency, the value of funds or resources at risk, the ability to reverse an action, the time needed to detect an error and the cost of correcting it.
In the consolidated text of the EU AI Act, requirements for high-risk systems include continuous risk management, appropriate levels of accuracy, robustness and cybersecurity, and effective human oversight. Oversight measures should be proportionate to the risk, degree of autonomy and context of use. Testing should use predefined metrics and thresholds appropriate to the intended purpose.
In business practice, this means assigning a specific person responsibility for approvals, monitoring and intervention. The interface should display sources, actions taken and the level of uncertainty, and the user must be able to stop the process. The risk of serious consequences from an error justifies restricting model permissions, adding further checks and extending shadow-mode testing. The regulatory classification should be assessed separately for the intended use and the organisation’s role.
13. How Can You Combine LLMs, SLMs and Rules in a Single Business Process?
Combining several technologies allows each stage of a process to be handled appropriately. For example, an SLM identifies the topic of a customer’s request, while an LLM drafts a response using the contact history and documents retrieved through RAG. If the case involves a refund, the system checks conditions and limits against established rules, then routes the operation for execution or approval by an authorised employee. This approach combines automation with oversight of decisions that have financial consequences.
At TTMS, we can help you assess where a similar solution would deliver the greatest benefit. We will start with the tasks that take up the most time: repetitive activities, searching for information or correcting errors. During a consultation, we will examine your workflows, available data and existing systems. Based on this assessment, we will recommend the technology and pilot scope, then work with you to define the expected outcomes and how to measure quality, time and costs.
Ready to choose your first process to improve? Book a consultation with us about implementing AI.
Sources
Eurostat, 20% of EU enterprises use AI technologies, 11 December 2025.
Fabrizio Dell’Acqua et al., Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality, Organization Science.
Erik Brynjolfsson, Danielle Li, Lindsey Raymond, Generative AI at Work, NBER Working Paper 31161, 2023.
Joel Becker, Nate Rush, Beth Barnes, David Rein, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR, 10 July 2025.
Microsoft Azure, What Are Small Language Models (SLMs)?
Microsoft Azure, Boost processing performance by combining AI models, 8 January 2025.
OpenAI, Enterprise privacy at OpenAI.
Google Cloud, The KPIs that actually matter for production AI agents, 26 February 2026.
NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024.
European Union, Regulation (EU) 2024/1689, consolidated text of 27 July 2026.
OWASP GenAI Security Project, LLM06:2025 Excessive Agency, 2025.
FAQ
Do You Need to Run Your Own AI Model to Work with Company Data?
Running your own model is one of several options. Companies can use an approved business environment, a managed API, a private cloud, dedicated infrastructure or a model running locally. The choice depends on data classification, processing location, retention, encryption, user identities, industry requirements and the agreement with the provider. For example, OpenAI states that business customer data is not used for training by default and offers additional retention controls for eligible API customers. Organisations should document the data flow for their specific configuration, as the model’s name does not describe the full security architecture.
Can RAG Replace Fine-Tuning a Model on Company Data?
RAG and fine-tuning address different needs. RAG retrieves current information from a controlled source when generating a response, making it well suited to knowledge bases, procedures, documentation and frequently updated content. Fine-tuning uses examples to adapt a model’s behaviour to a particular style, format or specialised task. AWS documentation comparing RAG and fine-tuning recommends starting with RAG for a question-answering system based on your own documents, particularly when up-to-date information and source references matter. Both approaches can work together when a process requires current knowledge and consistent behaviour within a specific domain.
Can a Single Process Use Both an LLM and an SLM?
Yes. A routing component can direct routine, clearly identified cases to an SLM and complex cases to a larger LLM. A third route passes cases to a person when the system detects missing data, low confidence or a high level of risk. Another option is to divide the process by function: an SLM classifies the document, an LLM prepares an explanation, code calculates values, and a workflow records the approved decision. This setup helps control costs and response times while retaining access to more advanced capabilities for difficult cases.
How Many Examples Do You Need for an LLM or SLM Pilot?
The number depends on the required measurement precision and how rare the errors are. When measuring the proportion of successful responses, a sample of approximately 385 independent cases provides an approximate margin of error of +/-5 percentage points at a 95% confidence level under conservative assumptions. A margin of +/-3 percentage points requires approximately 1,068 cases. The random sample should reflect the actual distribution of cases across languages, channels and user types, in proportion to their volumes. A separate test set should cover critical, edge and rare cases, along with attempts to manipulate the system.
How Often Should You Retest an Application Built on a Language Model?
A full evaluation should be run whenever the model, prompt version, tool, permissions, data source, business rules or input format changes. Production use also requires continuous monitoring of key performance indicators and regular regression testing. The frequency depends on the level of risk and how quickly the process changes. An application handling marketing content may follow a different schedule from a system supporting financial decisions. A practical approach is to run automated tests after every technical change, review trends monthly and conduct a business evaluation quarterly, with shorter cycles for high-risk applications.
Read more
