African financial institutions are sitting on a goldmine of data — transaction histories, mobile money flows, USSD session logs, and customer behaviour signals spanning millions of daily touchpoints. Yet the vast majority cannot use this data to train even a basic machine learning model. The reason is almost never a lack of data. It is almost always a lack of data readiness.
At Starks IT, we have helped over a dozen Nigerian fintechs and commercial banks scope, design, and build data infrastructure capable of supporting real-time AI workloads. Here is what we have learned.
The Three Endemic Problems
1. Schema Chaos at Ingestion
Most banks in Nigeria run on core banking systems that were installed in the 1990s or early 2000s — Oracle Flexcube, Temenos T24, Finacle. These systems emit data in proprietary formats, often flat files delivered via SFTP at end-of-day batch windows. When a bank wants to build a real-time fraud detection model, feeding 24-hour-old transaction records into it is simply not viable.
The fix begins with a Change Data Capture (CDC) layer sitting directly on the database transaction log — tools like Debezium on PostgreSQL or Oracle GoldenGate can stream individual row-level changes to a Kafka topic within milliseconds. This single architectural decision is often the most transformative thing we implement.
2. Data Quality Debt
A fintech that has been operating for three years likely has customer records created through four different onboarding flows, with inconsistent field naming, missing phone numbers in 18% of records, and date fields stored as VARCHAR. You cannot train a credit scoring model on data like this without serious remediation.
We recommend a data quality scoring framework applied at every pipeline stage — completeness, uniqueness, consistency, timeliness, and accuracy. Integrate this as a dbt test suite that runs on every transformation, and surface failures to the data engineering team before they reach the feature store.
3. Regulatory and Residency Constraints
NDPR 2.0 and CBN data residency guidelines create a complex web of compliance requirements. Customer PII cannot leave Nigerian borders. Certain financial data must be held for seven years. Derived analytics — model outputs, aggregated statistics — may have different treatment rules than raw data.
The practical implication is that a "send everything to S3 and figure it out later" approach will create a compliance liability. Data architecture must encode governance from day one: field-level encryption, column masking in the serving layer, and metadata tagging with retention policies.
The Reference Architecture We Use
After iterating across multiple engagements, we have converged on a four-layer architecture for African financial data pipelines:
- Ingestion Layer: Kafka + Debezium for real-time CDC; Airbyte for batch API sources; custom connectors for proprietary core banking file formats
- Landing Zone: MinIO (self-hosted S3-compatible object storage) deployed in a Nigerian data centre for raw data; immutable, append-only, partitioned by date
- Transformation Layer: dbt Core for SQL-based transformations; Great Expectations for data quality validation; Apache Spark for heavy processing jobs
- Serving Layer: Snowflake or BigQuery for analytical workloads; Redis Feature Store for low-latency ML feature serving; Metabase for business intelligence
Latency Targets That Actually Matter
Not every use case needs sub-second data freshness. A churn prediction model that runs weekly is perfectly well served by daily batch ingestion. But a real-time fraud detection system scoring individual transactions needs P99 latency under 200 milliseconds from event to model inference.
We work with clients to map their AI use case roadmap to latency requirements before touching any infrastructure. This prevents over-engineering for use cases that do not need it — and under-engineering for the ones that do.
What to Do First
If you are a CTO or Head of Data at a Nigerian financial institution trying to build AI capabilities, here is the sequence we recommend:
- Audit your existing data sources and document their format, latency, and quality characteristics
- Implement CDC on your core transaction database — even if you cannot use it immediately, start capturing the stream
- Define your first three AI use cases and their data requirements. Fraud detection, credit scoring, and churn prediction are the most common starting points
- Build the data quality framework before building models. A model trained on bad data will give you worse outcomes than no model at all
- Design your governance layer in parallel — retention policies, masking rules, access controls — not as an afterthought
Building an AI-ready data infrastructure is not a six-week project. It is typically an 18-to-24-month programme for a mid-sized bank. But the institutions that start now will have a durable competitive advantage over those that wait until they think they are "ready."