Limitations of AI in Legal Software: Risks of Incorrect Advice, Defective Court Filings and Missed Deadlines

• the most common AI errors in legal work,
• hallucinations involving legislation, judgments and case numbers,
• the procedural consequences of using incorrect AI-generated content in a court filing,
• a lawyer’s liability for advice prepared with the assistance of AI,
• principles for the safe implementation of AI software in a law firm.

An authentic case number and a fabricated proposition: lessons from the Supreme Administrative Court’s order in case I FZ 104/26

In its order of 23 June 2026 in case I FZ 104/26, the Supreme Administrative Court considered an appeal against an order refusing to stay the enforcement of a tax decision. The appellant was required to substantiate the conditions set out in Article 61 § 3 of the Law on Proceedings before Administrative Courts, particularly the risk of substantial damage or consequences that would be difficult to reverse. Counsel cited three judgments together with propositions purportedly drawn from them. The Court found that the judgments concerned different issues and that their written reasons did not contain the propositions attributed to them. The dates given for those judgments were also incorrect. The Supreme Administrative Court identified another issue: the submission was general in nature and did not contain specific data or documents that would have allowed the Court to assess the party’s financial position. The appeal was dismissed. The Court criticised counsel’s uncritical use of AI and emphasised that a client is entitled to expect professional service.

The decision illustrates three practical consequences of a defective workflow:

• a legal argument may lose the support of the source on which it purports to rely,
• a submission may omit facts and evidence that are material to the outcome,
• the court may question the reliability of counsel’s work.

A proper verification of a judgment includes checking the court, date, case number and type of decision, followed by reading the full written reasons. The lawyer should assess the factual context, legal basis, significance of the cited passage, available information about the decision’s finality and subsequent case law concerning the same issue.

Seven AI limitations that may affect the outcome of a case

1. A genuine source may be cited in support of a fabricated proposition

A model may provide a genuine case number, a provision of legislation or the title of a publication and then attribute content to that source which it does not contain. This type of error is more dangerous than an entirely fabricated case number because it may remain undetected during a cursory review.

The system should take the user directly to the relevant passage in the source. Verification by a lawyer remains necessary. A Stanford RegLab study of US legal research tools combining database searches with generated answers found hallucinations in 17-33% of responses. The study covered products and questions from the US market, so it does not measure the quality of Polish systems. Connecting a model to a legal database reduces the risk. Further safeguards are required to verify the accuracy of its output.

2. The model may apply an outdated or inapplicable provision

An answer may sound plausible while relying on a provision that has been amended, has not yet entered into force or applies in another jurisdiction. In a cross-border matter, the model may confuse the applicable law, jurisdiction and procedural rules.

An AI system for lawyers should identify the applicable legal system and the date as of which the law has been verified for each conclusion. The user should be able to see the version of the legislation, the date on which an amendment entered into force and the relevant official source. A label stating that the information is “up to date” has limited value unless the interface also indicates when and on what basis its currency was verified.

3. The model analyses only the case materials made available to it

The case file may be incomplete, an attachment may not have been read correctly, or a material fact may be contained in a table, scan or message that was not included in the analysis. The model formulates its answer on the basis of the materials it has received and has no knowledge of documents that were not submitted for analysis. As a result, a confident answer may be based on an incomplete picture of the case.

Before starting the analysis, the system should display a list of the files used, any processing errors and the scope of any omitted material. In matters requiring findings of fact, it is useful to provide a separate summary of assertions, supporting evidence and missing materials. This allows the lawyer to identify conclusions reached without access to a key document.

4. Calculating procedural deadlines requires clearly defined rules

The calculation of a deadline depends on factors including the type of proceedings, the method of service, the date of the relevant event, transitional provisions, public holidays and exceptions applicable to the specific procedural step. A language model may overlook one of these conditions or misread a date from a scanned document.

Procedural deadlines should be calculated using a controlled, rules-based mechanism with clearly stated input data. A model may extract dates from documents and suggest their legal significance. The final calculation should identify the legal basis, show how the deadline was calculated and be approved by the lawyer handling the case. Deadlines whose expiry may produce procedural consequences require an independent calculation and a second verification.

5. Factual assertions in an AI-assisted court filing require appropriate evidence

According to press reports discussing the judgment of the Regional Court in Wrocław of 27 November 2025 in case X GC 455/25, the claimant used ChatGPT when preparing the statement of claim. The claim was dismissed because the formal conditions of the request for proposals had not been satisfied. The Court also addressed the AI-generated content and the evidential value of the materials submitted. Printouts containing generated arguments, legal analysis and an assessment of the prospects of success did not constitute evidence of facts material to the determination of the case. They could be treated as part of the claimant’s legal argument.

A legal AI system should clearly distinguish between factual assertions, legal grounds and evidence. A missing contract, item of correspondence, proof of service or financial document remains an evidential gap regardless of the quality of the generated reasoning.

6. The model may reinforce the client’s assessment of the case

Clients often describe a dispute from one perspective and expect confirmation of their own assessment. A model may adopt the client’s assumptions and overlook the other party’s arguments. This creates a risk that the client will make a decision without understanding the weaknesses of the case.

The analysis should consider the opposing party’s perspective, including missing facts, potential counterarguments, procedural obstacles, alternative legal characterisations and the level of uncertainty. In client-facing communications, users should be able to refer a question to a lawyer, particularly when it concerns a deadline, a legal claim, criminal liability, termination of employment or a significant financial decision.

7. A convincing style may conceal errors and uncertainty

A model generates an answer word by word and may maintain the same professional tone when presenting both correct and incorrect conclusions. Assessing reliability on the basis of style is therefore unsafe. A confidence score expressed as a percentage may also create a false impression of precision if it has not been calibrated for the specific use case.

A well-designed system identifies its sources, missing information, conditions that could change the answer and situations in which it cannot provide a definitive conclusion. It should also be able to decline to answer and refer the matter to a lawyer.

Can a client claim that AI caused them to lose the case?

A client may raise such an allegation. The assessment of liability will depend on how the legal service was performed and on the circumstances of the individual case. In a contractual relationship, the primary rules are those set out in the Polish Civil Code. Under Article 355 § 2 of the Civil Code, the professional nature of the debtor’s activity must be taken into account when assessing due care. Article 471 of the Civil Code sets out the rules governing liability for non-performance or improper performance of an obligation.

An adverse outcome does not in itself establish that a legal services agreement was performed improperly. The assessment covers compliance with the applicable professional standard of care, the correct determination of the factual and legal position, and the appropriateness of the actions taken.

In a contractual liability dispute, the client should demonstrate:

1. the existence and terms of the obligation,
2. non-performance or improper performance of the obligation,
3. the loss suffered,
4. an adequate causal link between the breach and the loss.

Article 471 of the Civil Code establishes a presumption that non-performance or improper performance of an obligation results from circumstances for which the debtor is responsible. Whether the debtor can avoid liability must be assessed in light of the circumstances of the individual case. The assessment may be affected by how AI was used, particularly whether the sources were verified, the complete case file was considered, the current law was applied and control over the final content of the advice was retained. A system failure or an error attributable to the provider will be assessed together with the choice of tool, the contractual terms, the scope of testing performed and the lawyer’s method of verifying the output.

Examples include:

• missing a deadline because the date of service was determined incorrectly,
• advising against pursuing a legal remedy on the basis of an outdated provision,
• filing a submission containing a quotation that does not appear in the cited judgment,
• recommending a settlement on the basis of an incomplete case file or an incorrect calculation of the financial consequences,
• a chatbot giving the client a definitive answer without referring the matter to a lawyer.

These examples are illustrative. Any assessment of liability requires an examination of the scope of the engagement, the applicable standard of professional care, the loss suffered and the causal link. The conditions for disciplinary liability may be assessed separately. The Polish Law on the Bar and the Polish Act on Attorneys-at-Law also require advocates and attorneys-at-law to hold professional liability insurance.

Compulsory professional liability insurance does not necessarily cover every loss connected with the use of AI. The insurer’s liability depends on the terms of the policy and the circumstances of the individual event.

What procedural consequences may result from an incorrect AI-assisted court filing?

The consequences depend on the type of defect and the applicable procedural rules. An error resulting from the use of AI is assessed in the same way as any other error in a court filing. Its significance depends on how it affects compliance with formal requirements, proof of the relevant facts, the legal and factual basis of the relief sought, and compliance with applicable deadlines.

Depending on the type of defect, the consequences may include:

• the return of a filing if its formal defects have not been remedied,
• the rejection of a statement of claim, appeal or other means of challenge where the conditions specified in the applicable procedural rules are met,
• the court disregarding an application for evidence or the evidence itself,
• a finding that a material fact has not been proven,
• the court declining to accept an argument based on a source that does not support the proposition attributed to it,
• the dismissal of an application, appeal or claim because the required conditions have not been established,
• an order requiring the party to pay the costs of the proceedings or the imposition of a procedural sanction where provided for by the applicable rules.

In case I FZ 104/26, the Supreme Administrative Court dismissed the appeal because the appellant had failed to substantiate the conditions for staying the enforcement of the decision under Article 61 § 3 of the Law on Proceedings before Administrative Courts. The general nature of the arguments and the absence of supporting documents were relevant to this assessment. The incorrect references to case law were an additional factor in the Court’s critical assessment of how the submission had been prepared.

Any claim for damages against counsel is considered in separate proceedings. Disciplinary liability is assessed by the competent bodies of the relevant professional organisation.

What follows from professional rules and the AI Act in 2026?

On 15 June 2026, the Polish Bar Council announced the adoption of a resolution amending the Code of Ethics for Advocates and Dignity of the Profession by adding § 23e. According to the information published by the Polish Bar Council, technological tools should serve an auxiliary function. Their use must respect professional secrecy, the advocate’s independence and the advocate’s personal role in handling the case. The output produced by such a tool must be independently assessed and verified by the advocate.

For attorneys-at-law, relevant points of reference include the recommendations on the use of AI published by the Polish National Bar Council of Attorneys-at-Law. These practical guidelines address professional responsibility, confidentiality, output verification and human oversight.

At EU level, Article 4 of the AI Act has applied since 2 February 2025. It requires providers and deployers of AI systems to take measures supporting the development of AI literacy among their personnel. The measures selected should take account of the personnel’s technical knowledge, experience, education and training, as well as the context in which the system is used and the people in relation to whom it is intended to be used. The European Commission explains that the appropriate way to fulfil this obligation depends on the organisation’s role and the risks associated with the specific use of AI. Since 2 August 2026, the competent authorities have been responsible for supervising compliance with this obligation.

The classification of an AI system depends on its intended purpose. High-risk systems may include solutions intended to be used by, or on behalf of, a judicial authority to assist that authority in researching and interpreting facts and law and in applying the law to a specific set of facts. The assessment covers the system’s actual function, the intended purpose specified by the provider and the way in which the deployer uses it.

Law firm tools used to search documents, draft text or prepare summaries require an individual classification assessment. The fact that a system is used by a law firm or legal department does not in itself place it in the high-risk category. Following the amendments adopted in 2026, the obligations concerning systems listed in Annex III are due to apply from 2 December 2027.

Depending on the system’s function, the type of data involved and the way in which it is used, the GDPR, rules protecting professional secrecy, the applicable procedural rules, and civil and disciplinary liability rules may also apply.

How should an AI system for lawyers be designed to reduce risk?

A properly designed implementation should make it easier to identify errors, limit their effect on the matter being handled and document completion of the required review.

The scope of documentation should be proportionate to the risk. Audit logs, including the history of prompts and outputs, may contain information protected by professional secrecy. The organisation should define which data is recorded, who is authorised to access it, how long it is retained, how it is deleted and how the logs are secured. The full history of interactions with the system need not be retained where a narrower set of information is sufficient to demonstrate that the required review was performed.

A safe allocation of tasks between AI and the lawyer

Tasks should be assigned according to the potential harm and how easily an error can be detected.

This approach is consistent with the practical direction set out by the CCBE in its guide for lawyers: lawyers remain responsible for their work, advice and representations, and generative AI output must be reviewed before it is used.

A safe AI implementation in a law firm begins with process analysis

The first step is to identify where AI-generated output may affect advice given to a client, a court filing, the assessment of a document or the calculation of a deadline. This provides the basis for defining the appropriate data sources, access permissions, verification rules and the people responsible for approving the output.

If you are planning to use AI to analyse case files, work with documents or prepare contracts, explore the AI4Legal solution. We help law firms and legal departments design tools tailored to their workflows, security requirements and the scope of lawyers’ professional responsibilities.

FAQ: limitations of AI in legal software