Shaping the future of data systems and statistics: Key insights from India’s workshop on alternative data and frontier technology for policy

 


Official statistics have traditionally relied on standardized methods like censuses, surveys, and administrative records. However, the rapid digital transformation of economies, businesses, and societies has led to an explosion of new data sources. By integrating these with traditional data, and enabling innovative uses, we can begin to close persistent data gaps and generate real-time, granular insights. When combined with frontier technologies such as data science and AI, these innovations present a powerful opportunity to modernize the production of official statistics.  Achieving this, however, requires balancing tradition with innovation and addressing challenges around methodological rigor, data privacy, analytical capabilities, and data governance.

These opportunities and challenges formed the core of the discussions during the National Workshop on Using Alternate Data Sources and Frontier Technologies for Policy Making, held on June 5-6, in Delhi, India. The workshop, co-hosted by the Ministry of Statistics and Programme Implementation (MoSPI) and NITI Aayog, with the World Bank as a knowledge partner, brought together over 400 experts, including statisticians, data scientists, regulators, researchers, and policy professionals, from central and state governments, academia, think tanks, the private sector, and international organizations. 

The workshop featured four parallel technical tracks (discussed in the subsequent subchapters), each focusing on distinct data sources and technologies with potential for official statistics to improve statistical productivity, quality, and analytical insights to support evidence-based policymaking: 

MobilePhone Data (MPD): a new frontier for smarter policy

One of the most groundbreaking topics discussed was mobile positioning data, which is location data from mobile phone networks that shows how people move in real time. First used widely during the COVID-19 pandemic, to track mobility and support public health responses, MPD now has much broader applications. MPD offers valuable insights for disaster response, urban planning, transport infrastructure, migration, and tourism. For example, it can help monitor foot traffic at tourist destinations, identify travel routes, and assess how long people stay in different places—all in near real time. When combined with immigration, spending, and survey data, it creates a comprehensive tourism analytics toolkit to support policy and boost hospitality and retail sectors. 

The global use of MPD in official statistics is growing. Following Eurostat’s feasibility study in 2014, countries like Estonia, Finland, Spain, and Indonesia have adopted MPD to enhance tourism data. In 2024, Estonia’s central bank released the first national methodology for producing tourism statistics using MPDi.

Scannerdata: revolutionizing retail and price analysis

Another important workshop topic was retail scanner data, collected from barcode scanners at retail points of sale. The discussions explored how this data can modernize the way we understand consumer behavior and compile price indices. This data reveals what people are buying, where, in what quantities, and at what prices. It enables more frequent and detailed updates to Consumer Price Indices (CPI), which are vital for economic planning, wage setting, and monetary policy. As e-commerce grows, new techniques like web scraping through APIs offer even more digital price data—especially for goods like airfare, streaming services, or e-books that are now almost exclusively purchased online. 

In India, a large proportion of retail sale still takes place in small shops where transactions are recorded manually (if at all), and the final purchase price is determined through bargaining. However, the country’s organized retail and e-commerce sectors are expanding rapidly. Projections suggest they will account for around 20% of all retail sales in 2025 with a trend of further growth up to 34-38% in 2030. 

The workshop enabled MoSPI and state statisticians to engage with major retailers like Reliance and V-Mart, as well as regulators to plan a pilot for using scanner data and other alternate sources for price statistics.

Geospatialapplications: mapping the future

Geospatial applications were another highlight of the workshop. These include satellite imagery, GIS mapping, and drone-based data collection (including underwater drones), which are transforming the way we understand land use, population shifts, and environmental change. The workshop brought together statisticians from MoSPI, satellite images providers from the India Space Research Organization (ISRO), geospatial experts from academic and research institution in India and abroad, state statistical organizations and remote sensing agencies and GIS technologists from the private sector.

Visit Us 👇 

Website Link: https://statisticsaward.com

Nomination Link: https://statisticsaward.com/award-nomination

Registration Link: https://statisticsaward.com/award-registration

 

 

Comments

Post a Comment

Popular posts from this blog

Data experts race to preserve US government statistics amid quiet purges

Image
                                                After watching data sets be altered or disappear from U.S. government websites in unprecedented ways after President Donald Trump began his second term, an army of outside statisticians, demographers and computer scientists have joined forces to capture, preserve and share data sets, sometimes clandestinely. "There are trillions of bytes of data files, and I can't even imagine how many public dollars were spent to collect those data. ... But right now, they're sitting someplace that is inaccessible because there are no staff to appropriately manage those data," Jennifer Park, a study director for the Committee on National Statistics, National Academies of Sciences, Engineering, and Medicine, said during the conference hosted by the Association of Public Data Users (APDU). Their goal is to make sure they are available in t...
Read more

11 Essential Statistical Tools for Data-Driven Research

Image
  Best Statistical Tools, Statistical analysis is a crucial part of research, and statistical tools can streamline the process by helping researchers interpret the data in a simpler format. Here's the list of best statistical tools. 1. R In data analytics, R stands out among the top open-source statistical tools. Researchers in statistics use it. It provides high-quality toolboxes that can be used for many different things. The learning curve for the open-source programming language R is steep. R programming provides an efficient data handling and storage facility. R also has the best set available for array calculations. Thanks to its graphical tools for analysis, data visualization is another area where R shines. It is an all-inclusive high-level programming language with various functions, conditional loops, and decision expressions 2. Python Python is a versatile language with statistics modules. Its versatility and depth make it an excellent choice for creating analysis pipeli...
Read more

Trump Gets Rid of Those Pesky Statistics

Image
  For decades, Donald Trump has been at war with numbers. Some have capitulated more swiftly than others. His weight, his golf scores, and his net worth have long fallen in line. As I understand it, a Trump doctor appointment consists of going to a medical professional and announcing how much you would like to weigh and what your resting heart rate ought to be, and the wise doctor’s patriotic, good equipment cooperates to measure you correctly. (I have tried this myself without success. My scale is not a true patriot.) Mean, wicked scales that display unflattering numbers, and foolish, incompetent golf balls that do not traverse the correct distance, are promptly discarded and replaced with their more loyal counterparts. This is how value works! As Trump testified once in court, “My net worth fluctuates, and it goes up and down with the markets and with attitudes and with feelings, even my own feelings … Yes, even my own feelings, as to where the world is, where the world is...
Read more