Methodology and Quality Update

Methodology and Quality last update

29/10/2024

Statistical Presentation

Data description  

The Research and Development Survey presents data on the research and development in establishments in Saudi Arabia.
Research and Development Survey is a survey conducted to collect data on the basic characteristics as follows:
•    Study the volume of expenditure on R&D for all sectors.
•    Study the number of employees in the field of R&D for all sectors.
Data is also used to estimates:
•    Total R&D expenditure. 
•    Total number of employees in R&D.
•    Total numbers of researchers in the field of R&D.

Classification system

The following classifications are applied in the Industrial Production Survey:
The Frascati Manual for conducting research and development surveys:
It is a statistical classification based on the International Standard Industrial Classification of All Economic Activities (ISIC4) used to describe the productive activities of an establishment. 
It is a document that outlines the methodology for collecting statistics for research and development. This was prepared and published by the Organization for Economic Co-operation and Development (OECD).
The guide is available on the UNESCO website:  https://unesdoc.unesco.org/ark:/48223/pf0000227748_ara
The National Classification for Economic Activities (ISIC4):
It is a statistical classification based on the International Standard Industrial Classification of All Economic Activities (ISIC4) used to describe the productive activities of an establishment. 
Metadata are collected through interviews, so that outputs can be produces in accordance with all relevant classifications.
The classifications are available on the GASTAT’s website: https://www.stats.gov.sa/en/node

Sector coverage

Research and Development Survey covers all economic activities.
The size classification is divided into four categories in terms of size of the enterprise as follows:
•    Micro enterprises:
It includes all establishments in which the number of employees ranges between (1-5) employees.
•    Small enterprises:
It includes all establishments in which the number of employees ranges between (6-49) employees.
•    Medium enterprises:
It includes all establishments in which the number of employees ranges between (50-249) employees.
•    Large-sized:
It includes all establishments in which the number of employees is more than (249) employees.

Statistical concepts and definitions

Terminologies and concepts of the R&D Survey:
•    Research and development:
It includes every creative and systematic work undertaken with the aim of increasing 
the stock of knowledge, including knowledge of humanity, culture, and society, and in order to invent new applications of available knowledge.
•    Expenditure on research and development:
It includes all expenses related to research and development activities carried out within an
economic sector.
•    Employees of research and development: 
They are all individuals directly involved in research and development, as well as those providing direct services to R&D activities, such as R&D managers, administrative officers, technicians, and office staff. It excludes individuals who provide indirect support and assistance, such as restaurant staff, maintenance personnel, administrative, and security staff.
•    Researchers:
Professionals involved in designing and innovating new knowledge. Researchers conduct research and improve or develop concepts, theories, models, technologies, devices, software, or operating methods. Managers and administrators involved in planning and managing the scientific and technical aspects of research are also classified as "researchers". Doctoral students working in research and development should also be
counted as "researchers".
•    Technicians:
They are individuals whose main tasks require technical knowledge and expertise in one or more fields of engineering, physical sciences, life sciences (technicians), or social sciences, humanities, and the arts. They participate in research and development to carry out scientific and technical tasks that require the application of operational concepts and methods, as well as the use of research equipment, usually under the supervision of researchers.

Statistical unit

The statistical unit in the research and development survey is the establishment.

Statistical population

The statistical population for the research and development survey includes all establishments engaged in research and development in the Kingdom of Saudi Arabia.

Reference area

The survey sample is a representative sample for Saudi Arabia's 13 administrative regions.

Time coverage

Data are available from 2021-2023.

Base period

Not applicable. 

Unit of measure

Most results are measured by numbers (such as: Expenditure on R&D) (in thousands).

Reference period

References period to the variables or dataset as following:
R&D survey data is based on the fiscal year prior to the year the survey was conducted.

Confidentiality 

Confidentiality - policy

According to the Royal Decree No. 23 dated 07-12-1397, data must always be kept confidential and must be used by GASTAT only for statistical purposes.
Therefore, the data are protected in the data servers of the Authority.

Confidentiality - data treatment

Data of SMEs survey is presented in right tables in order to summarize, understand, as well as extract their results. Moreover, to compare them with other data, and to obtain statistical significance about the selected study population. However, referring to such data indicated in tables is much easier than going back to check the original questionnaire that may include some data like names and addresses of individuals, and names of data providers, which violates data confidentiality of statistical data.
“Anonymity of data” is one of the most important procedures. To keep data confidential,
GASTAT removed information on individual persons, households, or business entities such a way that the respondent cannot be identified either directly such as: (Names, addresses, contact numbers, or identification numbers. etc.) or indirectly (by combining different - especially rare - characteristics of respondents: (age, occupation, education etc.).

Release policy

Release calendar

Research and Development Survey has been included in the statistical calendar.

Release calendar access

The release calendar is available at:  https://www.stats.gov.sa/statistical-calendar-releases

User access

One of GASTAT’s objectives is to better meet its clients' needs, so it immediately provides them with the publication's results once the Research and Development Survey Publication is published.
 It also receives questions and inquiries of the clients about the Publication and its results through various communication channels, such as:
•    GASTAT official website:  www.stats.gov.sa
•    GASTAT official e-mail address:  info@stats.gov.sa
•    Client support e-mail:   info@stats.gov.sa
•    Official visits to GASTAT’s official head office in Riyadh or one of its branches in Saudi Arabia.
•    Official letters.
•    Statistical telephone: (199009).

Frequency of dissemination

Annual. 

Accessibility and clarity

News release

The announcements of each publication are available on release calendar as mentioned in 7.2. Release calendar access. The news release can be viewed on the website of GASTAT through the following link:
 https://stats.gov.sa/news

Publications

GASTAT issues Research and Development Survey publications and reports regularly within a pre-prepared dissemination plan and is published on GASTAT’s website. GASTAT is keen to publish its publications in a way that serves all users of different types, including publications in different formats that contain (publication tables, data graphs, indicators, methodology and quality report, and questionnaires) in both English and Arabic.
The results of the Research and Development Survey are available at:
https://www.stats.gov.sa/statistics

On-line database

The data is published on the statistical database at:
GASTAT (stats.gov.sa)

Micro-data access 

Microdata are unit-level datasets derived from surveys, censuses, and administrative records. These datasets provide detailed insights into individuals, households, businesses, and geographic areas, supporting the development of statistical indicators and in-depth research.
The different types of microdata files to meet different information needs:
•    Public use:
It consists sets of records containing information on individual persons, households, or business entities anonymized in such a way that the respondent cannot be identified either directly (by name, address, contact number, identity number etc.) or indirectly (by combining different - especially rare - characteristics of respondents: age, occupation, education etc.).
•    Scientific use:
These files established based on specific methodology asked by data requester to extract the datasets with specific characteristics used for strategic studies and decision making as well scientific research purposes on individuals, households and enterprises with no direct identifiers, which have been subject to control methods to protect confidentiality.
Access to Scientific Use Files (SUF) is restricted to authorized researchers who comply with ethical and confidentiality standards. Representative samples of SUF can be obtained through GASTAT's secure platform, "Etaha," while more sensitive datasets are accessible only through secure physical lab environments managed by GASTAT.

Other

Not available.

Documentation on methodology

•    Research and Development Survey framework: Concepts, definitions, issues and classifications are based on the 2008 SNA International Standards:
 https://www.stats.gov.sa/en/7055 
•    In addition to the Frascati Manual for conducting research and development surveys:
 https://unesdoc.unesco.org/ark:/48223/pf0000227748_ara

Quality documentation

Quality documentation covers documentation on methods and standards for assessing, measuring, and monitoring the quality of statistical process and output. It is based on standard quality criteria such as relevance, accuracy and reliability, timeliness and punctuality, accessibility and clarity, comparability, and coherence.

Quality management

Quality assurance

GASTAT declares that it considers the following principles: impartiality, user orientated, quality of processes and output, effectiveness of statistical processes, reducing the workload for respondents. 
Quality controls and validation of data are actions carried out throughout the process in different stages such as the data input and data collection and other final controls.

Quality assessment

GASTAT performs all statistical activities according to a national model (Generic Statistical Business Process Model – GSBPM). According to the GSBPM, the final phase of statistical activities is overall evaluation using information gathered in each phase or sub-process. This information is used to prepare the evaluation report which outlines all the quality issues related to the specific statistical activity and serves as input for improvement actions.

Relevance 

User needs 

Internal users in the GASTAT for the Research and Development Survey data:
•    National accounts.
Some several external users and beneficiaries greatly benefit from the Research and Development Survey data, including:
•    Government entities.
•    Regional and international organizations.
•    Research institutions.
•    Media.
•    Individuals.
The disseminated key variables that used by external users:

User satisfaction

Not available.

Completeness 

The R&D survey data is based on two main sources in order to provide comprehensive information on the volume of funding and expenditure on R&D as well as the numbers of employees (researchers-technicians-and other staff), and the status of the data is complete.

Accuracy and reliability 

Overall accuracy

•    The data collected is improved through the researchers, that have been selected according to a set of practical and objective criteria and training program related to the field of work.
•    Alert, prevention, and correction rules are applied during the data collection process on the electronic questionnaire for the Research and Development Survey to improve data quality.
•    Data is checked with previous years to identify any significant changes in the data.
•    The internal consistency of the data is checked before it is finalized.
•    The links between variables are checked and coherence between different data series is confirmed.

Timeliness and punctuality 

Timeliness 

The General Authority for Statistics is committed to applying internationally approved standards for publishing statistics, including the timing standard issued by the European Statistical Organization. It announces and clarifies the publication dates of statistics on its official website through the statistical calendar and adheres to the announced dates. In the event of any delay, an update will be provided.

Punctuality

The publication is done according to the publication dates in the statistical calendar published for the Research and Development Survey on the website page of the General Authority for Statistics.
The data are available at the expected time, as scheduled in the statistical release calendar, If the publication is delayed, reasons shall be provided.

Coherence and comparability

Comparability - geographical

Data are fully comparable.

Comparability - over time 

The survey began in 2021 as an annual survey.

Coherence- cross domain 

Not applicable. 

Coherence - sub annual and annual statistics 

Not applicable. 

Coherence- National Accounts 

Not applicable. 

Coherence - internal 

The Research and Development Survey estimates have full internal coherence, as they are all based on the same corpus of microdata, and they are calculated using the same estimation methods.

Resources used 

Data revision

Data revision - policy

Not applicable, only final results will be published.

Data revision - practice

Not applicable, only final results will be published.

Statistical processing

Source data

The research and development survey data are based on two sources:
First source: A comprehensive inventory of the survey framework, the 2022 business framework has been used. It is a basic framework for this survey and other economic research to be conducted by GASTAT in the future, through which all establishments are classified by economic activity and by the size of the establishment at the level of the Kingdom and administrative regions. The activities of these establishments are classified according to the National Classification of Economic Activities. (ISIC-4)
 The disseminated key variables of the research and development survey data are :
•    Research and Development funding
•    Expenditure on R&D.
•    Number of R&D employees.
•    Number of R&D Researchers.
Second source: From the administrative records of the Government Entities.
The main published variables from the administrative data source are:
•    Research and Development funding
•    Expenditure on research and development.
•    Number of R&D employees. 
•    Number of R&D Researchers.

Frequency of data collection 

Annual.

Data collection

Data collection from the survey:
Data for the Business Confidence Index Survey is collected through Computer-Assisted Telephone Interviews (CATI), Computer-Assisted Web Interviews (CAWI).
Data collection from administrative records:
In coordination with GASTAT's relevant departments involved in conducting the survey and managing data collection, the administrative data of the publication of Research and Development Survey is obtained from the Ministry of Finance, which includes R&D expenditure data. 
The data is stored in the authority's databases after undergoing auditing and review processes following approved statistical methods and recognized quality standards. If errors or discrepancies are discovered, the data is cross-referenced with the data source for correction or clarification.

Data validation

Data are reviewed and matched to ensure their accuracy and precision in a way that suits their nature with the aim of giving the presented statistics quality and accuracy. 
The data of the current year publication are compared with the data of the previous year to ensure their integrity and consistency in preparation for processing data and extracting and reviewing results.
In addition to the data processing and tabulation to check their accuracy, all the outputs are stored and uploaded to the database after being calculated by GASTAT to be reviewed and processed by specialists in Research and Development Survey through modern technologies and software designed for this purpose.

Data compilation

Data Coding:
Interviewers in Research and Development Survey collect from respondents, a detailed description of each field. This information is then coded in-house by an automated process, which is reviewed by a small-dedicated team of coding experts using a series of consistency checks.
Data editing:
Specialists of business, investment, and international trade statistics department have processed and analyzed data in this stage, and this step was based on the following measures:
•    Sorting and arranging data in groups or different categories in a serial order.
•    Summarizing detailed data into key points or data.
•    Combining many data segments and ensuring their interconnection.
•    Processing incomplete or missing data.
•    Processing illogical data.
•    Converting data into statistically significant data.
•    Arranging, presenting, and interpreting data.
Compensation (for non-response cases or incomplete datasets): 
The approach used for compensation in Research and Development Survey, whether for establishments with incomplete response or missing data for specific variables. Reinterviews are allowed to obtain missing data from non-respondents. Subsequently, dealing with missing or non-response data involves assessing it by following a scientific approach to estimate the results based on considerations such as historical data series and an acceptable range of missing data, along with estimates built on class-level data. 
Extrapolation and weighting: 
After processing the data collected from respondents, survey weights were generated to produce indicator tables by following two main steps in creating survey weights: 
•    Adjustment of non-response.
•    Calibration weight
Applied statistical estimation:
GASTAT has relied on the formulas approved by the international standards in calculating the key indicators for Research and Development Survey., as follows:
•    Total R&D expenditure = total government sector expenditure + total private sector expenditure + total education sector expenditure
•    Total R&D employees = Total employees in the government sector + Total employees in the private sector + Total employees in the education sector
•    Total R&D researchers = Total researchers in the government sector + Total researchers in the private sector + Total researchers in the education sector

Adjustment

Not applicable, only final results will be published.