Companies accumulate data at a pace that far outstrips their ability to read it. Transactions, browsing logs, sensors, support tickets: every interaction leaves a trail. The problem is no longer having data, but finding within it something worth knowing. Data mining is the discipline that extracts, from large volumes of information, useful, comprehensible and previously unknown patterns. It is not about querying what we already know is there, but about discovering relationships that no one had framed as a question.
This article walks through the fundamental techniques of data mining—clustering, association rules, classification—their place within a proven methodology, the most profitable commercial use cases and the regulatory framework that conditions their use when working with personal data.
What data mining is and is not
Data mining is the modelling stage within a broader process known as KDD (Knowledge Discovery in Databases): the discovery of knowledge from data. It is worth separating it from neighbouring concepts. It is not the same as descriptive analytics (reports and dashboards that summarise what has happened), nor the simple querying of a database. Mining looks for generalisable patterns: structures that recur and that allow you to describe or predict future behaviour. It is, to a large extent, the practical application of machine learning to business problems with real data.
Unsupervised learning: clustering
Clustering groups similar records without anyone having previously labelled the categories. The algorithm discovers on its own the latent structure of the data. Its star application is customer segmentation: instead of imposing segments a priori, you let the data reveal the natural groups of behaviour.
The best-known algorithm is k-means, which divides the data into k groups by minimising the distance of each point to its centroid; it is fast and scalable, but it requires setting k in advance and assumes spherically shaped groups. Hierarchical clustering builds a tree of groupings (a dendrogram) without requiring a prior k, at the cost of higher computational expense. DBSCAN, based on density, discovers arbitrarily shaped groups and isolates noise, which makes it ideal for anomaly detection. A classic business technique is RFM analysis (Recency, Frequency, Monetary value), which segments customers according to when they last bought, how often they buy and how much they spend.
Association rules: market basket analysis
Association rules discover relationships of the type "those who buy A tend to buy B". The paradigmatic case is market basket analysis, which examines which products appear together in baskets in order to design promotions, recommendations and in-store placement. Three metrics govern the quality of a rule:
| Metric | What it measures | Interpretation |
|---|---|---|
| Support | How frequently the itemset appears | Is the rule relevant in volume? |
| Confidence | Probability of B given A | Is the rule reliable? |
| Lift | Association versus chance | Lift > 1 indicates a real association, not coincidence |
Lift is especially important: a high confidence can be misleading if product B is very popular in its own right. Only when the lift exceeds 1 is there a genuine association beyond what would be expected by chance. The Apriori and FP-Growth algorithms are the classic methods for extracting these rules efficiently.
Supervised learning: classification and prediction
When we do have labelled examples, classification learns to assign categories to new records: predicting whether a customer will churn, whether a transaction is fraudulent or whether an email is spam. Decision trees and random forests stand out for their interpretability and robustness; logistic regression remains the standard for binary problems; and gradient boosting methods dominate competitions for their accuracy on tabular data. Prediction (regression) estimates continuous values, such as future demand or customer lifetime value.
The CRISP-DM process: from problem to deployment
Data mining is not throwing an algorithm at a table. The industry-standard methodology, CRISP-DM (Cross-Industry Standard Process for Data Mining), organises the work into six iterative phases:
- Business understanding: translate the business objective into a data problem.
- Data understanding: explore, profile and assess the quality of the available information.
- Data preparation: cleaning, integration and transformation. It is the longest phase and usually consumes the bulk of the effort.
- Modelling: apply and tune the algorithms.
- Evaluation: validate that the model answers the business objective, not just a technical metric.
- Deployment: put the model into production and monitor it.
The most repeated rule of thumb in the sector is that data preparation represents the greater part of the work. A brilliant model on dirty data produces brilliant, false conclusions.
Personal data: GDPR and the AI Act
When the mined data identifies individuals, the General Data Protection Regulation (GDPR) applies directly. Its principles of data minimisation and purpose limitation clash with the temptation to "mine everything just in case": only the data necessary for a legitimate and declared purpose should be processed. The GDPR also regulates automated decisions with significant effects (Article 22) and requires clear legal bases. In Spain, the Spanish Data Protection Agency (AEPD) has published specific guidance on the use of profiling techniques.
To this framework is added the European Artificial Intelligence Regulation (AI Act), which classifies systems by risk level and imposes obligations of transparency, data governance and human oversight on high-risk ones. Data mining that feeds credit-scoring or recruitment-selection models falls squarely within its scope. A responsible mining programme incorporates these requirements from the design stage, not as a later patch.
Use cases that generate returns
Data mining proves its value in concrete, measurable business problems. Churn prediction identifies in advance the customers most likely to leave, which makes it possible to act with targeted retention before losing them. Fraud detection leverages anomaly-detection techniques to flag transactions that deviate from the account holder's usual pattern. Recommendation systems, fed by clustering and association rules, suggest relevant products and raise the average order value. And demand forecasting adjusts inventory to real behaviour, reducing both stockouts and the capital tied up in the warehouse.
The factor that separates a profitable project from a sterile experiment is not the sophistication of the algorithm, but the clear connection to a decision the company can make. A churn model only adds value if there is a retention process to act on it; a segmentation only helps if marketing is going to treat each group differently. That is why CRISP-DM begins with business understanding and not with data: the most interesting pattern is useless if no one can act on it. Before modelling, it is worth asking what will be done with the result, what threshold justifies an intervention and how impact will be measured, so that the discovery leads to an action and not to a report that no one reads.
Common mistakes
- Confusing correlation with causation: two variables moving together does not imply that one causes the other.
- Overfitting: a model that memorises the noise of the historical data and fails with new data.
- Data leakage: including in the model variables that would not be available at the real moment of prediction.
- Mining personal data without a legal basis, with the consequent exposure to sanctions.
- Skipping data preparation and modelling on dirty or biased information.
Frequently asked questions
Are data mining and machine learning the same thing? They are closely related but not identical. Data mining is the process of discovering useful patterns in large volumes; machine learning provides many of the algorithms that process uses. Mining focuses on business knowledge; machine learning, on the predictive model.
How much data is needed to start? It depends on the technique and the complexity of the pattern. More than volume, what is decisive is the quality, the representativeness and the absence of bias in the data.
What distinguishes clustering from classification? Clustering is unsupervised: it discovers groups without prior labels. Classification is supervised: it learns from already-labelled examples to assign known categories.
Is consent needed to segment customers? It depends on the legal basis and the purpose. Profiling with personal data must rest on a valid GDPR basis and the data subject must be informed; a prior assessment is advisable when the processing is high-risk.
Conclusion
Data mining turns inert files into decisions: it reveals which customers resemble one another, which products travel together in the basket and which signals anticipate a defection or a fraud. But its value depends less on the algorithm of the moment than on the rigour of the process: a well-framed business objective, clean and representative data, a validation that distinguishes the real pattern from the statistical mirage, and regulatory compliance that respects the GDPR and the AI Act from the very first data point. The organisation that masters that rigour turns its historical data into an advantage that competitors cannot copy, because it is made of its own data. At Summum Artificial Intelligence we design data-mining projects that start from the business problem and end in deployed, auditable and compliant models.