Big Data

Intermediate

Big data refers to extremely large and complex sets of data that cannot easily be stored, processed or analysed with traditional tools, often described by their volume, velocity and variety.

The growth of digital activity, sensors, online transactions and social media has created vast amounts of data. Big data is often described using three Vs: volume, the sheer amount of data; velocity, the speed at which data are generated and must be processed; and variety, the many different formats, from structured tables to text, images and video. Some add veracity, the reliability of data, and value.

Analysing big data requires specialized technologies, such as distributed computing, cloud storage and machine learning.

For example, a payments company might analyse billions of transactions in real time to detect fraud patterns, or a retailer might combine purchase histories, website behaviour and location data to personalise offers.

In finance, big data is used for fraud detection, credit scoring, algorithmic trading, risk management, customer analysis and regulatory compliance. It also raises concerns about privacy, security, bias in automated decisions and the risk of finding misleading patterns in large data sets.

For worked examples and full lessons, visit notesbysara.org.