# kb

id: big-data-20260505-6
file: data/en.wikipedia.org/wiki/Big_data-6.md
category: reference
title: Big data
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Big_data
date_saved: 2026-05-05T09:53:36.506913+00:00
instance: kb-cron
parent: big-data-20260505-0
summary:
=== Science === The Large Hadron Collider experiments represent about 150 million sensors delivering data 40 million times per second. There are nearly 600 million collisions per second. After filtering and refraining from recording more than 99.99995% of these streams, there are 1,000 collisions of interest per second. As a result, only working with less than 0.001% of the sensor stream data, the data flow from all four LHC experiments represents 25 petabytes annual rate before replication (as of 2012). This becomes nearly 200 petabytes after replication. If all sensor data were recorded in LHC, the data flow would be extremely hard to work with. The data flow would exceed 150 million petabytes annual rate, or nearly 500 exabytes per day, before replication. To put the number in perspective, this is equivalent to 500 quintillion (5×1020) bytes per day, almost 200 times more than all the other sources combined in the world. The Square Kilometre Array is a radio telescope built of thousands of antennas. It is expected to be operational by 2024. Collectively, these antennas are expected to gather 14 exabytes and store one petabyte per day. It is considered one of the most ambitious scientific projects ever undertaken.


id: big-data-20260505-7
file: data/en.wikipedia.org/wiki/Big_data-7.md
category: reference
title: Big data
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Big_data
date_saved: 2026-05-05T09:53:36.506913+00:00
instance: kb-cron
parent: big-data-20260505-0
summary:
== Research activities == Encrypted search and cluster formation in big data were demonstrated in March 2014 at the American Society of Engineering Education. Gautam Siwach engaged at Tackling the challenges of Big Data by MIT Computer Science and Artificial Intelligence Laboratory and Amir Esmailpour at the UNH Research Group investigated the key features of big data as the formation of clusters and their interconnections. They focused on the security of big data and the orientation of the term towards the presence of different types of data in an encrypted form at cloud interface by providing the raw definitions and real-time examples within the technology. Moreover, they proposed an approach for identifying the encoding technique to advance towards an expedited search over encrypted text leading to the security enhancements in big data. In March 2012, The White House announced a national "Big Data Initiative" that consisted of six federal departments and agencies committing more than $200 million to big data research projects. The initiative included a National Science Foundation "Expeditions in Computing" grant of $10 million over five years to the AMPLab at the University of California, Berkeley. The AMPLab also received funds from DARPA, and over a dozen industrial


id: big-data-20260505-8
file: data/en.wikipedia.org/wiki/Big_data-8.md
category: reference
title: Big data
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Big_data
date_saved: 2026-05-05T09:53:36.506913+00:00
instance: kb-cron
parent: big-data-20260505-0
summary:
=== Critiques of the big data paradigm === "A crucial problem is that we do not know much about the underlying empirical micro-processes that lead to the emergence of the[se] typical network characteristics of Big Data." In their critique, Snijders, Matzat, and Reips point out that often very strong assumptions are made about mathematical properties that may not at all reflect what is really going on at the level of micro-processes. Mark Graham has leveled broad critiques at Chris Anderson's assertion that big data will spell the end of theory: focusing in particular on the notion that big data must always be contextualized in their social, economic, and political contexts. Even as companies invest eight- and nine-figure sums to derive insight from information streaming in from suppliers and customers, less than 40% of employees have sufficiently mature processes and skills to do so. To overcome this insight deficit, big data, no matter how comprehensive or well analyzed, must be complemented by "big judgment", according to an article in the Harvard Business Review. Much in the same line, it has been pointed out that the decisions based on the analysis of big data are inevitably "informed by the world as it


id: big-data-20260505-9
file: data/en.wikipedia.org/wiki/Big_data-9.md
category: reference
title: Big data
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Big_data
date_saved: 2026-05-05T09:53:36.506913+00:00
instance: kb-cron
parent: big-data-20260505-0
summary:
=== Critiques of big data execution === Ulf-Dietrich Reips and Uwe Matzat wrote in 2014 that big data had become a "fad" in scientific research. Researcher danah boyd has raised concerns about the use of big data in science neglecting principles such as choosing a representative sample by being too concerned about handling the huge amounts of data. This approach may lead to results that have a bias in one way or another. Integration across heterogeneous data resources—some that might be considered big data and others not—presents formidable logistical as well as analytical challenges, but many researchers argue that such integrations are likely to represent the most promising new frontiers in science. In the provocative article "Critical Questions for Big Data", the authors title big data a part of mythology: "large data sets offer a higher form of intelligence and knowledge [...], with the aura of truth, objectivity, and accuracy". Users of big data are often "lost in the sheer volume of numbers", and "working with Big Data is still subjective, and what it quantifies does not necessarily have a closer claim on objective truth". Recent developments in BI domain, such as pro-active reporting especially target improvements in the usability


id: collocation-remote-sensing-20260505-0
file: data/en.wikipedia.org/wiki/Collocation_(remote_sensing)-0.md
category: reference
title: Collocation (remote sensing)
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Collocation_(remote_sensing)
date_saved: 2026-05-05T09:53:37.734675+00:00
instance: kb-cron
parent: collocation-remote-sensing-20260505-0
summary:
Collocation is a procedure used in remote sensing to match measurements from two or more different instruments. This is done for two main reasons: for validation purposes when comparing measurements of the same variable, and to relate measurements of two different variables either for performing retrievals or for prediction. In the second case the data is later fed into some type of statistical inverse method such as an artificial neural network, statistical classification algorithm, kernel estimator or a linear least squares. In principle, most collocation problems can be solved by a nearest neighbor search, but in practice there are many other considerations involved and the best method is highly specific to the particular matching of instruments. Here we deal with some of the most important considerations along with specific examples. There are at least two main considerations when performing collocations. The first is the sampling pattern of the instrument. Measurements may be dense and regular, such as those from a cross-track scanning satellite instrument. In this case, some form of interpolation may be appropriate. On the other hand, the measurements may be sparse, such as a one-off field campaign designed for some particular validation exercise. The second consideration is the


id: collocation-remote-sensing-20260505-1
file: data/en.wikipedia.org/wiki/Collocation_(remote_sensing)-1.md
category: reference
title: Collocation (remote sensing)
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Collocation_(remote_sensing)
date_saved: 2026-05-05T09:53:37.734675+00:00
instance: kb-cron
parent: collocation-remote-sensing-20260505-0
summary:
== Interpolation == For gridded data such as assimilation or reanalysis data, interpolation is likely the most appropriate method for performing any type of comparison. A specific point in both physical position and time is easy to locate within the grid and interpolation performed between the nearest neighbors. Linear interpolation (bilinear, trilinear etc.) is the most common, though cubic is used as well but is probably not worth the extra computational overhead. If the variable of interest has a relatively smooth rate of change (temperature is a good example of this because it has a diffusion mechanism, radiative transfer, not available to other atmospheric variables), then interpolation can eliminate much of the error associated with collocation. Interpolation may also be appropriate for many types of satellite instruments, for instance a cross-track scanning instrument like Landsat. In data derived from the Advanced Microwave Sounding Unit (AMSU) are interpolated (although not for the purposes of collocation) using a slight variation of trilinear interpolation. Since measurements within a single scan track are laid out in an approximately rectangular grid, bilinear interpolation can be performed. By searching for the nearest overlapping scan track both forwards and backwards in time, the spatial interpolates can then


id: combinatorial-data-analysis-20260505-0
file: data/en.wikipedia.org/wiki/Combinatorial_data_analysis-0.md
category: reference
title: Combinatorial data analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Combinatorial_data_analysis
date_saved: 2026-05-05T09:53:38.930340+00:00
instance: kb-cron
parent: combinatorial-data-analysis-20260505-0
summary:
In statistics, combinatorial data analysis (CDA) is the study of data sets where the order in which objects are arranged is important. CDA can be used either to determine how well a given combinatorial construct reflects the observed data, or to search for a suitable combinatorial construct that does fit the data. == See also == Cluster analysis Geometric data analysis Structured data analysis (statistics) Seriation (statistics) == References ==


id: concept-drift-20260505-0
file: data/en.wikipedia.org/wiki/Concept_drift-0.md
category: reference
title: Concept drift
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Concept_drift
date_saved: 2026-05-05T09:53:40.159460+00:00
instance: kb-cron
parent: concept-drift-20260505-0
summary:
In predictive analytics, data science, machine learning and related fields, concept drift or drift is an evolution of data that invalidates the data model. It happens when the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways. This causes problems because the predictions become less accurate as time passes. Drift detection and drift adaptation are of paramount importance in the fields that involve dynamically changing data and data models. == Predictive model decay == In machine learning and predictive analytics this drift phenomenon is called concept drift. In machine learning, a common element of a data model are the statistical properties, such as probability distribution of the actual data. If they deviate from the statistical properties of the training data set, then the learned predictions may become invalid, if the drift is not addressed. == Data configuration decay == Another important area is software engineering, where three types of data drift affecting data fidelity may be recognized. Changes in the software environment ("infrastructure drift") may invalidate software infrastructure configuration. "Structural drift" happens when the data schema changes, which may invalidate databases. "Semantic drift" is changes in the meaning of data


id: concept-drift-20260505-1
file: data/en.wikipedia.org/wiki/Concept_drift-1.md
category: reference
title: Concept drift
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Concept_drift
date_saved: 2026-05-05T09:53:40.159460+00:00
instance: kb-cron
parent: concept-drift-20260505-0
summary:
=== Remedy methods === DDM (Drift Detection Method): detects drift by monitoring the model's error rate over time. When the error rate passes a set threshold, it enters a warning phase, and if it passes another threshold, it enters a drift phase. EDDM (Early Drift Detection Method): improves DDM's detection rate by tracking the average distance between two errors instead of only the error rate. ADWIN (Adaptive Windowing): dynamically stores a window of recent data and warns the user if it detects a significant change between the statistics of the window's earlier data compared to more recent data. KSWIN (Kolmogorov–Smirnov Windowing): detects drift based on the Kolmogorov-Smirnov statistical test. DDM and EDDM: Concept Drift Detection online supervised methods that rely on sequential error monitoring to estimate the evolving error rate. ADWIN and KSWIN: Windowing maintain a "window", a subset of the most recent data, of the data stream, which it checks for statistical differences across the window. == Applications in security == Concept drift is a recurring issue in security analytics, especially in malware and intrusion detection. In these systems, models are often trained on past logs, binaries or network traces, but the behaviour of attackers changes over time as


id: concept-drift-20260505-2
file: data/en.wikipedia.org/wiki/Concept_drift-2.md
category: reference
title: Concept drift
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Concept_drift
date_saved: 2026-05-05T09:53:40.159460+00:00
instance: kb-cron
parent: concept-drift-20260505-0
summary:
=== Concept Drift, Concept Evolution, and Adversarial Manipulation === Concept drift refers to when the relationship between input features and their corresponding labels changes gradually over time. Consider a simple example: phishing emails. The older kinds of attacks would leverage prominent signals, such as the presence of the word "lottery," while newer variants would introduce more polished phrasing, such as "lottery and not scam," in attempts to appear more legitimate and evade detection. This is a form of concept drift because the underlying class—phishing—remains the same, but the feature patterns that define it shift as attackers adapt. It is important, though, to distinguish concept drift from its related but inherently different challenge: concept evolution.“Concept evolution” occurs when an entirely new attack family emerges with feature patterns the model has never encountered. The model is exposed to new features and patterns that were not present in the training data. In the multi-class setting, it means a completely new label is introduced. For the binary setting (benign vs malicious), a completely new malicious family is introduced; however, the model still tries to make predictions based on outdated feature patterns. One such example would be if a model is trained with both trojan


id: concept-drift-20260505-3
file: data/en.wikipedia.org/wiki/Concept_drift-3.md
category: reference
title: Concept drift
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Concept_drift
date_saved: 2026-05-05T09:53:40.159460+00:00
instance: kb-cron
parent: concept-drift-20260505-0
summary:
==== Real ==== USP Data Stream Repository, 27 real-world stream datasets with concept drift compiled by Souza et al. (2020). Access Airline, approximately 116 million flight arrival and departure records (cleaned and sorted) compiled by E. Ikonomovska. Reference: Data Expo 2009 Competition [1]. Access Chess.com (online games) and Luxembourg (social survey) datasets compiled by I. Zliobaite. Access ECUE spam 2 datasets each consisting of more than 10,000 emails collected over a period of approximately 2 years by an individual. Access from S.J.Delany webpage Elec2, electricity demand, 2 classes, 45,312 instances. Reference: M. Harries, Splice-2 comparative evaluation: Electricity pricing, Technical report, The University of South Wales, 1999. Access from J.Gama webpage. Comment on applicability. PAKDD'09 competition data represents the credit evaluation task. It is collected over a five-year period. Unfortunately, the true labels are released only for the first part of the data. Access Sensor stream and Power supply stream datasets are available from X. Zhu's Stream Data Mining Repository. Access SMEAR is a benchmark data stream with a lot of missing values. Environment observation data over 7 years. Predict cloudiness. Access Text mining, a collection of text mining datasets with concept drift, maintained by I. Katakis. Access Gas Sensor Array


id: continuous-analytics-20260505-0
file: data/en.wikipedia.org/wiki/Continuous_analytics-0.md
category: reference
title: Continuous analytics
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Continuous_analytics
date_saved: 2026-05-05T09:53:41.378945+00:00
instance: kb-cron
parent: continuous-analytics-20260505-0
summary:
Continuous analytics is a data science process that abandons ETLs and complex batch data pipelines in favor of cloud-native and microservices paradigms. Continuous data processing enables real time interactions and immediate insights with fewer resources. == Defined == Analytics is the application of mathematics and statistics to big data. Data scientists write analytics programs to look for solutions to business problems, like forecasting demand or setting an optimal price. The continuous approach runs multiple stateless engines which concurrently enrich, aggregate, infer and act on the data. Data scientists, dashboards and client apps all access the same raw or real-time data derivatives with proper identity-based security, data masking and versioning in real-time. Traditionally, data scientists have not been part of IT development teams, like regular Java programmers. This is because their skills set them apart in their own department not normally related to IT, i.e., math, statistics, and data science. So it is logical to conclude that their approach to writing software code does not enjoy the same efficiencies as the traditional programming team. In particular traditional programming has adopted the Continuous Delivery approach to writing code and the agile methodology. That releases software in a continuous circle, called iterations. Continuous


id: cosine-similarity-20260505-0
file: data/en.wikipedia.org/wiki/Cosine_similarity-0.md
category: reference
title: Cosine similarity
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Cosine_similarity
date_saved: 2026-05-05T09:53:42.619881+00:00
instance: kb-cron
parent: cosine-similarity-20260505-0
summary:
In data analysis, cosine similarity is a measure of similarity between two non-zero vectors defined in an inner product space. Cosine similarity is the cosine of the angle between the vectors; that is, it is the dot product of the vectors divided by the product of their lengths. It follows that the cosine similarity does not depend on the magnitudes of the vectors, but only on their angle. The cosine similarity always belongs to the interval [ − 1 , + 1 ] . {\displaystyle [-1,+1].} For example, two proportional vectors have a cosine similarity of +1, two orthogonal vectors have a similarity of 0, and two opposite vectors have a similarity of −1. In some contexts, the component values of the vectors cannot be negative, in which case the cosine similarity is bounded in [ 0 , 1 ] {\displaystyle [0,1]} . For example, in information retrieval and text mining, each word is assigned a different coordinate and a document is represented by the vector of the numbers of occurrences of each word in the document. Cosine similarity then gives a useful measure of how similar two documents are likely to be, in terms of their subject matter, and


id: cosine-similarity-20260505-1
file: data/en.wikipedia.org/wiki/Cosine_similarity-1.md
category: reference
title: Cosine similarity
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Cosine_similarity
date_saved: 2026-05-05T09:53:42.619881+00:00
instance: kb-cron
parent: cosine-similarity-20260505-0
summary:
angular distance = D θ := arccos ⁡ ( cosine similarity ) π = θ π {\displaystyle {\text{angular distance}}=D_{\theta }:={\frac {\arccos({\text{cosine similarity}})}{\pi }}={\frac {\theta }{\pi }}} angular similarity = S θ := 1 − angular distance = 1 − θ π {\displaystyle {\text{angular similarity}}=S_{\theta }:=1-{\text{angular distance}}=1-{\frac {\theta }{\pi }}} Or, if the vector elements are always positive: angular distance = D θ := 2 ⋅ arccos ⁡ ( cosine similarity ) π = 2 θ π {\displaystyle {\text{angular distance}}=D_{\theta }:={\frac {2\cdot \arccos({\text{cosine similarity}})}{\pi }}={\frac {2\theta }{\pi }}} angular similarity = S θ := 1 − angular distance = 1 − 2 θ π {\displaystyle {\text{angular similarity}}=S_{\theta }:=1-{\text{angular distance}}=1-{\frac {2\theta }{\pi }}} Unfortunately, computing the inverse cosine (arccos) function is slow, making the use of the angular distance more computationally expensive than using the more common (but not metric) cosine distance above. === L2-normalized Euclidean distance === Another effective proxy for cosine distance can be obtained by L 2 {\displaystyle L_{2}} normalisation of the vectors, followed by the application of normal Euclidean distance. Using this technique each term in each vector is first divided by the magnitude of the vector, yielding a vector of unit length. Then the Euclidean distance over


id: cosine-similarity-20260505-2
file: data/en.wikipedia.org/wiki/Cosine_similarity-2.md
category: reference
title: Cosine similarity
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Cosine_similarity
date_saved: 2026-05-05T09:53:42.619881+00:00
instance: kb-cron
parent: cosine-similarity-20260505-0
summary:
cos ⁡ ( ∠ A C − ∠ C B ) ≥ cos ⁡ ( ∠ A B ) ≥ cos ⁡ ( ∠ A C + ∠ C B ) . {\displaystyle \cos(\angle {AC}-\angle {CB})\geq \cos(\angle {AB})\geq \cos(\angle {AC}+\angle {CB}).} Using the cosine addition and subtraction formulas, these two inequalities can be written in terms of the original cosines, cos ⁡ ( A , C ) ⋅ cos ⁡ ( C , B ) + ( 1 − cos ⁡ ( A , C ) 2 ) ⋅ ( 1 − cos ⁡ ( C , B ) 2 ) ≥ cos ⁡ ( A , B ) , {\displaystyle \cos(A,C)\cdot \cos(C,B)+{\sqrt {\left(1-\cos(A,C)^{2}\right)\cdot \left(1-\cos(C,B)^{2}\right)}}\geq \cos(A,B),} cos ⁡ ( A , B ) ≥ cos ⁡ ( A , C ) ⋅ cos ⁡ ( C , B ) − ( 1 − cos ⁡ ( A , C ) 2 ) ⋅ ( 1 − cos ⁡ ( C , B ) 2 ) . {\displaystyle \cos(A,B)\geq \cos(A,C)\cdot \cos(C,B)-{\sqrt {\left(1-\cos(A,C)^{2}\right)\cdot \left(1-\cos(C,B)^{2}\right)}}.} This form of the triangle inequality can be used to bound the minimum and maximum similarity of two objects A and B if the similarities to a reference


id: cross-industry-standard-process-for-data-mining-20260505-0
file: data/en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining-0.md
category: reference
title: Cross-industry standard process for data mining
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining
date_saved: 2026-05-05T09:53:43.884915+00:00
instance: kb-cron
parent: cross-industry-standard-process-for-data-mining-20260505-0
summary:
The Cross-industry standard process for data mining, known as CRISP-DM, is an open standard process model that describes common approaches used by data mining experts. It is the most widely-used analytics model. In 2015, IBM released a new methodology called Analytics Solutions Unified Method for Data Mining/Predictive Analytics (also known as ASUM-DM), which refines and extends CRISP-DM. == History == CRISP-DM was conceived in 1996 and became a European Union project under the ESPRIT funding initiative in 1997. The project was led by five companies: Integral Solutions Ltd (ISL), Teradata, Daimler AG, NCR Corporation, and OHRA, an insurance company. This core consortium brought different experiences to the project. ISL, was later acquired and merged into SPSS. The computer giant NCR Corporation produced the Teradata data warehouse and its own data mining software. Daimler-Benz had a significant data mining team. OHRA was starting to explore the potential use of data mining. The first version of the methodology was presented at the 4th CRISP-DM SIG Workshop in Brussels in March 1999, and published as a step-by-step data mining guide later that year. Between 2006 and 2008, a CRISP-DM 2.0 SIG was formed, and there were discussions about updating the CRISP-DM process model.


id: dark-data-20260505-0
file: data/en.wikipedia.org/wiki/Dark_data-0.md
category: reference
title: Dark data
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Dark_data
date_saved: 2026-05-05T09:53:45.047576+00:00
instance: kb-cron
parent: dark-data-20260505-0
summary:
Dark data is data which is acquired through various computer network operations but not used in any manner to derive insights or for decision making. The ability of an organisation to collect data can exceed the throughput at which it can analyse the data. In some cases the organisation may not even be aware that the data is being collected. IBM estimate that roughly 90 percent of data generated by sensors and analog-to-digital conversions never get used. In an industrial context, dark data can include information gathered by sensors and telematics. Organizations retain dark data for a multitude of reasons, and it is estimated that most companies are only analyzing 1% of their data. Often it is stored for regulatory compliance and record keeping. Some organizations believe that dark data could be useful to them in the future, once they have acquired better analytic and business intelligence technology to process the information. Because storage is inexpensive, storing data is easy. However, storing and securing the data usually entails greater expenses (or even risk) than the potential return profit. In academic discourse, the term dark data was essentially coined by Bryan P. Heidorn. He uses it to describe research data, especially


id: data-access-20260505-0
file: data/en.wikipedia.org/wiki/Data_access-0.md
category: reference
title: Data access
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_access
date_saved: 2026-05-05T09:53:46.267466+00:00
instance: kb-cron
parent: data-access-20260505-0
summary:
Data access is a generic term referring to a process which has both an IT-specific meaning and other connotations involving access rights in a broader legal and/or political sense. In the former it typically refers to software and activities related to storing, retrieving, or acting on data housed in a database or other repository. == Details == Two fundamental types of data access exist: sequential access (as in magnetic tape, for example) random access (as in indexed media) Data access crucially involves authorization to access different data repositories. Data access can help distinguish the abilities of administrators and users. For example, administrators may have the ability to remove, edit and add data, while general users may not even have "read" rights if they lack access to particular information. Historically, each repository (including each different database, file system, etc.), might require the use of different methods and languages, and many of these repositories stored their content in different and incompatible formats. Over the years standardized languages, methods, and formats, have developed to serve as interfaces between the often proprietary, and always idiosyncratic, specific languages and methods. Such standards include SQL (1974- ), ODBC (ca 1990- ), JDBC, XQJ, ADO.NET, XML, XQuery,


id: data-analysis-for-fraud-detection-20260505-0
file: data/en.wikipedia.org/wiki/Data_analysis_for_fraud_detection-0.md
category: reference
title: Data analysis for fraud detection
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_analysis_for_fraud_detection
date_saved: 2026-05-05T09:53:47.474137+00:00
instance: kb-cron
parent: data-analysis-for-fraud-detection-20260505-0
summary:
Fraud represents a significant problem for governments and businesses and specialized analysis techniques for discovering fraud using them are required. Some of these methods include knowledge discovery in databases (KDD), data mining, machine learning and statistics. They offer applicable and successful solutions in different areas of electronic fraud crimes. In general, the primary reason to use data analytics techniques is to tackle fraud since many internal control systems have serious weaknesses. For example, the currently prevailing approach employed by many law enforcement agencies to detect companies involved in potential cases of fraud consists in receiving circumstantial evidence or complaints from whistleblowers. As a result, a large number of fraud cases remain undetected and unprosecuted. In order to effectively test, detect, validate, correct error and monitor control systems against fraudulent activities, businesses entities and organizations rely on specialized data analytics techniques such as data mining, data matching, the sounds like function, regression analysis, clustering analysis, and gap analysis. Techniques used for fraud detection fall into two primary classes: statistical techniques and artificial intelligence. == Statistical techniques == Examples of statistical data analysis techniques are: Data preprocessing techniques for detection, validation, error correction, and filling up of missing or incorrect data. Calculation


id: data-analysis-for-fraud-detection-20260505-1
file: data/en.wikipedia.org/wiki/Data_analysis_for_fraud_detection-1.md
category: reference
title: Data analysis for fraud detection
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_analysis_for_fraud_detection
date_saved: 2026-05-05T09:53:47.474137+00:00
instance: kb-cron
parent: data-analysis-for-fraud-detection-20260505-0
summary:
Early data analysis techniques were oriented toward extracting quantitative and statistical data characteristics. These techniques facilitate useful data interpretations and can help to get better insights into the processes behind the data. Although the traditional data analysis techniques can indirectly lead us to knowledge, it is still created by human analysts. To go beyond, a data analysis system has to be equipped with a substantial amount of background knowledge, and be able to perform reasoning tasks involving that knowledge and the data provided. In effort to meet this goal, researchers have turned to ideas from the machine learning field. This is a natural source of ideas, since the machine learning task can be described as turning background knowledge and examples (input) into knowledge (output). If data mining results in discovering meaningful patterns, data turns into information. Information or patterns that are novel, valid and potentially useful are not merely information, but knowledge. One speaks of discovering knowledge, before hidden in the huge amount of data, but now revealed. The machine learning and artificial intelligence solutions may be classified into two categories: 'supervised' and 'unsupervised' learning. These methods seek for accounts, customers, suppliers, etc. that behave 'unusually' in order to output


id: data-exploration-20260505-0
file: data/en.wikipedia.org/wiki/Data_exploration-0.md
category: reference
title: Data exploration
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_exploration
date_saved: 2026-05-05T09:53:48.638463+00:00
instance: kb-cron
parent: data-exploration-20260505-0
summary:
Data exploration is an approach similar to initial data analysis, whereby a data analyst uses visual exploration to understand what is in a dataset and the characteristics of the data, rather than through traditional data management systems. These characteristics can include size or amount of data, completeness of the data, correctness of the data, possible relationships amongst data elements or files/tables in the data. Data exploration is typically conducted using a combination of automated and manual activities. Automated activities can include data profiling or data visualization or tabular reports to give the analyst an initial view into the data and an understanding of key characteristics. This is often followed by manual drill-down or filtering of the data to identify anomalies or patterns identified through the automated actions. Data exploration can also require manual scripting and queries into the data (e.g. using languages such as SQL or R) or using spreadsheets or similar tools to view the raw data. All of these activities are aimed at creating a mental model and understanding of the data in the mind of the analyst, and defining basic metadata (statistics, structure, relationships) for the data set that can be used in further analysis. Once this


id: data-farming-20260505-0
file: data/en.wikipedia.org/wiki/Data_farming-0.md
category: reference
title: Data farming
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_farming
date_saved: 2026-05-05T09:53:49.876847+00:00
instance: kb-cron
parent: data-farming-20260505-0
summary:
Data farming is the process of using designed computational experiments to “grow” data, which can then be analyzed using statistical and visualization techniques to obtain insight into complex systems. These methods can be applied to any computational model. Data farming differs from Data mining, as the following metaphors indicate: Miners seek valuable nuggets of ore buried in the earth, but have no control over what is out there or how hard it is to extract the nuggets from their surroundings. ... Similarly, data miners seek to uncover valuable nuggets of information buried within massive amounts of data. Data-mining techniques use statistical and graphical measures to try to identify interesting correlations or clusters in the data set. Farmers cultivate the land to maximize their yield. They manipulate the environment to their advantage using irrigation, pest control, crop rotation, fertilizer, and more. Small-scale designed experiments let them determine whether these treatments are effective. Similarly, data farmers manipulate simulation models to their advantage, using large-scale designed experimentation to grow data from their models in a manner that easily lets them extract useful information. ...the results can reveal root cause-and-effect relationships between the model input factors and the model responses, in addition to rich


id: data-fusion-20260505-0
file: data/en.wikipedia.org/wiki/Data_fusion-0.md
category: reference
title: Data fusion
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_fusion
date_saved: 2026-05-05T09:53:51.162435+00:00
instance: kb-cron
parent: data-fusion-20260505-0
summary:
Data fusion is the process of integrating multiple data sources to produce more consistent, accurate, and useful information than that provided by any individual data source. Data fusion processes are often categorized as low, intermediate, or high, depending on the processing stage at which fusion takes place. Low-level data fusion combines several sources of raw data to produce new raw data. The expectation is that fused data is more informative and synthetic than the original inputs. For example, sensor fusion is also known as (multi-sensor) data fusion and is a subset of information fusion. The concept of data fusion has origins in the evolved capacity of humans and animals to incorporate information from multiple senses to improve their ability to survive. For example, a combination of sight, touch, smell, and taste may indicate whether a substance is edible. == The JDL/DFIG model == In the mid-1980s, the Joint Directors of Laboratories formed the Data Fusion Subpanel (which later became known as the Data Fusion Group). With the advent of the World Wide Web, data fusion thus included data, sensor, and information fusion. The JDL/DFIG introduced a model of data fusion that divided the various processes. Currently, the six levels with


id: data-fusion-20260505-1
file: data/en.wikipedia.org/wiki/Data_fusion-1.md
category: reference
title: Data fusion
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_fusion
date_saved: 2026-05-05T09:53:51.162435+00:00
instance: kb-cron
parent: data-fusion-20260505-0
summary:
== For enhanced contextual awareness == With a multitude of built-in sensors including motion sensor, environmental sensor, position sensor, a modern mobile device typically gives mobile applications access to a number of sensory data which could be leveraged to enhance the contextual awareness. Using signal processing and data fusion techniques such as feature generation, feasibility study and principal component analysis (PCA) such sensory data will greatly improve the positive rate of classifying the motion and contextual relevant status of the device. Many context-enhanced information techniques are provided by Snidaro, et al. == Statistical methods == === Bayesian auto-regressive Gaussian processes === Gaussian processes are a popular machine learning model. If an auto-regressive relationship between the data is assumed, and each data source is assumed to be a Gaussian process, this constitutes a non-linear Bayesian regression problem. === Semiparametric estimation === Many data fusion methods assume common conditional distributions across several data sources. Recently, methods have been developed to enable efficient estimation within the resulting semiparametric model. == See also == Data assimilation Data munging Image fusion Information integration Integrative level Meta-analysis Sensor fusion == References == === Sources === General references Hall, Dave L.; Llinas, James (1997). "Introduction to Multisensor


id: data-profiling-20260505-0
file: data/en.wikipedia.org/wiki/Data_profiling-0.md
category: reference
title: Data profiling
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_profiling
date_saved: 2026-05-05T09:53:52.402806+00:00
instance: kb-cron
parent: data-profiling-20260505-0
summary:
Data profiling is the process of examining the data available from an existing information source (e.g. a database or a file) and collecting statistics or informative summaries about that data. The purpose of these statistics may be to: Find out whether existing data can be easily used for other purposes Improve the ability to search data by tagging it with keywords, descriptions, or assigning it to a category Assess data quality, including whether the data conforms to particular standards or patterns Assess the risk involved in integrating data in new applications, including the challenges of joins Discover metadata of the source database, including value patterns and distributions, key candidates, foreign-key candidates, and functional dependencies Assess whether known metadata accurately describes the actual values in the source database Understanding data challenges early in any data intensive project, so that late project surprises are avoided. Finding data problems late in the project can lead to delays and cost overruns. Have an enterprise view of all data, for uses such as master data management, where key data is needed, or data governance for improving data quality. == Introduction == Data profiling refers to the analysis of information for use in a data warehouse


id: data-science-20260505-0
file: data/en.wikipedia.org/wiki/Data_science-0.md
category: reference
title: Data science
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_science
date_saved: 2026-05-05T09:53:53.540719+00:00
instance: kb-cron
parent: data-science-20260505-0
summary:
Data science is an interdisciplinary academic field that uses statistics, scientific computing, scientific methods, processing, scientific visualization, algorithms, and systems to extract or extrapolate knowledge from potentially noisy, structured, or unstructured data. Data science plays a critical role in modern decision-making by enabling organizations to extract actionable insights from large and complex datasets. Data science also integrates domain knowledge from the underlying application domain (e.g., natural sciences, information technology, and medicine). Data science is multifaceted and can be described as a science, a research paradigm, a research method, a discipline, a workflow, and a profession. Data science is "a concept to unify statistics, data analysis, informatics, and their related methods" to "understand and analyze actual phenomena" with data. It uses techniques and theories drawn from many fields within the context of mathematics, statistics, computer science, information science, and domain knowledge. However, data science is distinct from computer science and information science. Turing Award winner Jim Gray imagined data science as a "fourth paradigm" of science (empirical, theoretical, computational, and now data-driven) and asserted that "everything about science is changing because of the impact of information technology" and the data deluge. Data science is often described as a multidisciplinary and rapidly


id: data-science-20260505-1
file: data/en.wikipedia.org/wiki/Data_science-1.md
category: reference
title: Data science
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_science
date_saved: 2026-05-05T09:53:53.540719+00:00
instance: kb-cron
parent: data-science-20260505-0
summary:
data collection and integration; data cleaning and preparation (handling missing values, outliers, encoding, normalisation); feature engineering and selection; visualisation and descriptive statistics; fitting and evaluating statistical or machine-learning models; communicating results and ensuring reproducibility (e.g., reports, notebooks, and dashboards). Lifecycle frameworks such as CRISP-DM describe these steps from business understanding through deployment and monitoring. Data science involves working with larger datasets that often require advanced computational and statistical methods to analyze. Data scientists often work with unstructured data such as text or images and use machine learning algorithms to build predictive models. Data science often uses statistical analysis, data preprocessing, and supervised learning. Recent studies indicate that AI is moving towards data-centric approaches, focusing on the quality of datasets rather than just improving AI models. This trend focuses on improving system performance by cleaning, refining, and labeling data (Bhatt et al., 2024). As AI systems grow larger, the data-centric view has become increasingly important. == Cloud computing for data science == Cloud computing can offer access to large amounts of computational power and storage. In big data, where volumes of information are continually generated and processed, these platforms can be used to handle complex and resource-intensive analytical tasks. Some distributed


id: data-version-control-20260505-0
file: data/en.wikipedia.org/wiki/Data_version_control-0.md
category: reference
title: Data version control
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data_version_control
date_saved: 2026-05-05T09:53:54.761385+00:00
instance: kb-cron
parent: data-version-control-20260505-0
summary:
Data version control is a method of working with data sets. It is similar to the version control systems used in traditional software development, but is optimized to allow better processing of data and collaboration in the context of data analytics, research, and any other form of data analysis. Data version control may also include specific features and configurations designed to facilitate work with large data sets and data lakes. == History == === Background === As early as 1985, researchers recognized the need for defining timing attributes in database tables, which would be necessary for tracking changes to databases. This research continued into the 1990s, and the theory was formalized into practical methods for managing data in relational databases, providing some of the foundational concepts for what would later become data version control. In the early 2010s the size of data sets was rapidly expanding, and relational databases were no longer sufficient to manage the amounts of data organizations were accumulating. The rise of the Apache Hadoop eco system, with HDFS as a storage layer, and later object storage had become dominant in big data operations. Research into data management tools and data version control systems increased sharply, along


id: data-driven-model-20260505-0
file: data/en.wikipedia.org/wiki/Data-driven_model-0.md
category: reference
title: Data-driven model
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data-driven_model
date_saved: 2026-05-05T09:53:55.946685+00:00
instance: kb-cron
parent: data-driven-model-20260505-0
summary:
Data-driven models are a class of computational models that primarily rely on historical data collected throughout a system's or process' lifetime to establish relationships between input, internal, and output variables. Commonly found in numerous articles and publications, data-driven models have evolved from earlier statistical models, overcoming limitations posed by strict assumptions about probability distributions. These models have gained prominence across various fields, particularly in the era of big data, artificial intelligence, and machine learning, where they offer valuable insights and predictions based on the available data. == Background == These models have evolved from earlier statistical models, which were based on certain assumptions about probability distributions that often proved to be overly restrictive. The emergence of data-driven models in the 1950s and 1960s coincided with the development of digital computers, advancements in artificial intelligence research, and the introduction of new approaches in non-behavioural modelling, such as pattern recognition and automatic classification. == Key Concepts == Data-driven models encompass a wide range of techniques and methodologies that aim to intelligently process and analyse large datasets. Examples include fuzzy logic, fuzzy and rough sets for handling uncertainty, neural networks for approximating functions, global optimization and evolutionary computing, statistical learning theory, and Bayesian


id: data-informed-decision-making-20260505-0
file: data/en.wikipedia.org/wiki/Data-informed_decision-making-0.md
category: reference
title: Data-informed decision-making
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Data-informed_decision-making
date_saved: 2026-05-05T09:53:57.178692+00:00
instance: kb-cron
parent: data-informed-decision-making-20260505-0
summary:
Data-informed decision-making (DIDM) refers to the collection and analysis of data to guide decisions and improve chances of success. Another form of this process is referred to as data-driven decision-making, "which is defined similarly as making decisions based on hard data as opposed to intuition, observation, or guesswork." DIDM is used in education communities, where data is used with the goal of helping students and improving curricula, among other fields in which data is used to inform decisions. While "data based decision-making" is a more common term, "data-informed decision-making" is the preferred term, since decisions should not be based solely on quantitative data. Data-driven decision-making is commonly used in the context of business growth and entrepreneurship. Many educators have access to some type of a data system for analyzing their students' data. These data systems present data to educators in an over-the-counter data format (embedding labels, supplemental documentation, and a help system, making key package/display and content decisions) to improve the success of educators' data-informed decision-making. In business, fostering and actively supporting data-driven decision-making in their firm and among their colleagues may be one of the central responsibilities of CIOs (Chief Information Officers) or CDOs (Chief Data Officers). Assessment in


id: dataset-shift-20260505-0
file: data/en.wikipedia.org/wiki/Dataset_shift-0.md
category: reference
title: Dataset shift
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Dataset_shift
date_saved: 2026-05-05T09:53:59.453907+00:00
instance: kb-cron
parent: dataset-shift-20260505-0
summary:
Dataset shift is a phenomenon in machine learning and statistics in which the joint distribution of input variables and target labels is different in the training phase and the deployment or test phase (i.e., P t r a i n ( X , Y ) ≠ P t e s t ( X , Y ) {\displaystyle P_{train}(X,Y)\neq P_{test}(X,Y)} ). This happens when the statistical properties of data used to train a model are no longer representative of the data encountered in real-world use, often resulting in degraded predictive performance and diminished generalization ability. Dataset shift is a generic term for a number of particular types of distributional change. Covariate shift is when the distribution of the input features changes, but the conditional relationship between inputs and outputs remains constant . Prior probability shift (or label shift) happens when the distribution of target labels changes, but the conditional distribution of inputs given labels stays the same. Concept shift (also known as concept drift) is the change of the conditional relationship between inputs and outputs that renders previously learned patterns invalid over time. A key challenge for deploying machine learning systems is dataset shift, in particular in dynamic environments where the


id: degree-rips-bifiltration-20260505-0
file: data/en.wikipedia.org/wiki/Degree-Rips_bifiltration-0.md
category: reference
title: Degree-Rips bifiltration
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Degree-Rips_bifiltration
date_saved: 2026-05-05T09:54:00.644610+00:00
instance: kb-cron
parent: degree-rips-bifiltration-20260505-0
summary:
The degree-Rips bifiltration is a simplicial filtration used in topological data analysis for analyzing the shape of point cloud data. It is a multiparameter extension of the Vietoris–Rips filtration that possesses greater stability to data outliers than single-parameter filtrations, and which is more amenable to practical computation than other multiparameter constructions. Introduced in 2015 by Lesnick and Wright, the degree-Rips bifiltration is a parameter-free and density-sensitive vehicle for performing persistent homology computations on point cloud data. == Definition == It is standard practice in topological data analysis (TDA) to associate a sequence of nested simplicial complexes to a finite data set in order to detect the persistence of topological features over a range of scale parameters. One way to do this is by considering the sequence of Vietoris–Rips complexes of a finite set in a metric space indexed over all scale parameters. If X {\displaystyle X} is a finite set in a metric space, then this construction is known as the Vietoris–Rips (or simply "Rips") filtration on X {\displaystyle X} , commonly denoted Rips ( X ) {\displaystyle {\text{Rips}}(X)} or R ( X ) {\displaystyle {\mathcal {R}}(X)} . The Rips filtration can be expressed as a functor Rips ( X


id: directional-component-analysis-20260505-0
file: data/en.wikipedia.org/wiki/Directional_component_analysis-0.md
category: reference
title: Directional component analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Directional_component_analysis
date_saved: 2026-05-05T09:54:01.826255+00:00
instance: kb-cron
parent: directional-component-analysis-20260505-0
summary:
Directional component analysis (DCA) is a statistical method used in climate science for identifying representative patterns of variability in space-time data-sets such as historical climate observations, weather prediction ensembles or climate ensembles. The first DCA pattern is a pattern of weather or climate variability that is both likely to occur (measured using likelihood) and has a large impact (for a specified linear impact function, and given certain mathematical conditions: see below). The first DCA pattern contrasts with the first PCA pattern, which is likely to occur, but may not have a large impact, and with a pattern derived from the gradient of the impact function, which has a large impact, but may not be likely to occur. DCA differs from other pattern identification methods used in climate research, such as EOFs, rotated EOFs and extended EOFs in that it takes into account an external vector, the gradient of the impact. DCA provides a way to reduce large ensembles from weather forecasts or climate models to just two patterns. The first pattern is the ensemble mean, and the second pattern is the DCA pattern, which represents variability around the ensemble mean in a way that takes impact into account. DCA contrasts


id: directional-component-analysis-20260505-1
file: data/en.wikipedia.org/wiki/Directional_component_analysis-1.md
category: reference
title: Directional component analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Directional_component_analysis
date_saved: 2026-05-05T09:54:01.826255+00:00
instance: kb-cron
parent: directional-component-analysis-20260505-0
summary:
The two axes represent anomalies of annual mean rainfall at two locations, with the highest total rainfall anomaly values towards the top right corner of the diagram The joint variability of the rainfall anomalies at the two locations is assumed to follow a bivariate normal distribution The ellipse shows a single contour of probability density from this bivariate normal, with higher values inside the ellipse The red dot at the centre of the ellipse shows zero rainfall anomalies at both locations The blue parallel-line arrow shows the principal axis of the ellipse, which is also the first PCA spatial pattern vector In this case, the PCA pattern is scaled so that it touches the ellipse The diagonal straight line shows a line of constant positive total rainfall anomaly, assumed to be at some fairly extreme level The red dotted-line arrow shows the first DCA pattern, which points towards the point at which the diagonal line is tangent to the ellipse In this case, the DCA pattern is scaled so that it touches the ellipse From this diagram, the DCA pattern can be seen to possess the following properties: Of all the points on the diagonal line, it is the one


id: directional-change-intrinsic-time-20260505-0
file: data/en.wikipedia.org/wiki/Directional-change_intrinsic_time-0.md
category: reference
title: Directional-change intrinsic time
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Directional-change_intrinsic_time
date_saved: 2026-05-05T09:54:03.000900+00:00
instance: kb-cron
parent: directional-change-intrinsic-time-20260505-0
summary:
Directional-change intrinsic time is an event-based operator to dissect a data series into a sequence of alternating trends of defined size δ {\displaystyle \delta } . The directional-change intrinsic time operator was developed for the analysis of financial market data series. It is an alternative methodology to the concept of continuous time. Directional-change intrinsic time operator dissects a data series into a set of drawups and drawdowns or up and down trends that alternate with each other. An established trend comes to an end as soon as a trend reversal is observed. A price move that extends a trend is called overshoot and leads to new price extremes. Figure 1 provides an example of a price curve dissected by the directional change intrinsic time operator. The frequency of directional-change intrinsic events maps (1) the volatility of price changes conditional to (2) the selected threshold δ {\displaystyle \delta } . The stochastic nature of the underlying process is mirrored in the non-equal number of intrinsic events observed over equal periods of physical time. Directional-change intrinsic time operator is a noise filtering technique. It identifies regime shifts, when trend changes of a particular size occur and hides price fluctuations that are smaller


id: doacross-parallelism-20260505-0
file: data/en.wikipedia.org/wiki/DOACROSS_parallelism-0.md
category: reference
title: DOACROSS parallelism
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/DOACROSS_parallelism
date_saved: 2026-05-05T09:54:04.174776+00:00
instance: kb-cron
parent: doacross-parallelism-20260505-0
summary:
DOACROSS parallelism is a parallelization technique used to perform Loop-level parallelism by utilizing synchronisation primitives between statements in a loop. This technique is used when a loop cannot be fully parallelized by DOALL parallelism due to data dependencies between loop iterations, typically loop-carried dependencies. The sections of the loop which contain loop-carried dependence are synchronized, while treating each section as a parallel task on its own. Therefore, DOACROSS parallelism can be used to complement DOALL parallelism to reduce loop execution times. == Description == DOACROSS parallelism is particularly useful when one statement depends on the values generated by another statement. In such a loop, DOALL parallelism can not be implemented in a straightforward manner. If the first statement blocks the execution of the second statement until the required value has been produced, then the two statements would be able to execute independent of each other (i.e.), each of the aforementioned statements would be parallelized for simultaneous execution using DOALL parallelism. The following pseudocode illustrates the operation of DOACROSS parallelism in such a situation. === Example === In this example, each iteration of the loop requires the value written into a by the previous iteration. However, the entire statement is not


id: document-oriented-database-20260505-0
file: data/en.wikipedia.org/wiki/Document-oriented_database-0.md
category: reference
title: Document-oriented database
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Document-oriented_database
date_saved: 2026-05-05T09:54:05.454091+00:00
instance: kb-cron
parent: document-oriented-database-20260505-0
summary:
A document-oriented database, or document store, is a computer program and data storage system designed for storing, retrieving, and managing document-oriented information, also known as semi-structured data. Document-oriented databases are one of the main categories of NoSQL databases, and the popularity of the term "document-oriented database" has grown alongside the adoption of NoSQL itself. XML databases are a subclass of document-oriented databases optimized for XML documents. Graph databases are similar, but add another layer, the relationship, which allows them to link documents for rapid traversal. Document-oriented databases are conceptually an extension of the key–value store, another type of NoSQL database. In key-value stores, data is treated as opaque by the database, whereas document-oriented systems exploit the internal structure of documents to extract metadata and optimize storage and queries. Although in practice the distinction can be minimal due to modern tooling, document stores are designed to provide a richer programming experience with modern programming techniques. Document databases differ significantly from traditional relational databases (RDBs). Relational databases store data in predefined tables, often requiring an object to be split across multiple tables. In contrast, document databases store all information for a given object in a single document, with each document potentially having


id: document-oriented-database-20260505-1
file: data/en.wikipedia.org/wiki/Document-oriented_database-1.md
category: reference
title: Document-oriented database
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Document-oriented_database
date_saved: 2026-05-05T09:54:05.454091+00:00
instance: kb-cron
parent: document-oriented-database-20260505-0
summary:
=== Relationship to relational databases === In a relational database, data is organized into predefined types represented as tables. Each table contains rows (records) with a fixed set of columns (fields), so all records in a table share the same structure. Administrators typically define indexes on selected fields to improve query performance. A central principle of relational database design is database normalization, in which data that might otherwise be repeated is stored in separate tables and linked using keys. When records in different tables are related, a foreign key is used to associate them. For example, an address book application may store a contact's name, image, phone numbers, mailing addresses, and email addresses. In a normalized relational design, separate tables might be created for contacts, phone numbers, and email addresses. The phone number table would include a foreign key referencing the associated contact. To reconstruct a complete contact record, the database retrieves related information from each table using the foreign keys and combines it into a single record. In contrast, a document-oriented database stores all data related to an object within a single document, and stored in the database as a single entry. In the address book example,the contact's name,


id: doe-mean-plot-20260505-0
file: data/en.wikipedia.org/wiki/DOE_mean_plot-0.md
category: reference
title: DOE mean plot
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/DOE_mean_plot
date_saved: 2026-05-05T09:54:06.621514+00:00
instance: kb-cron
parent: doe-mean-plot-20260505-0
summary:
In statistics, specifically the analysis of data in the design of experiments, a DOE mean plot is a graphical tool used to analyze data from an experiment. In it, it demonstrates the relative importance of all the factors in an experiment with respect to a chosen location statistic, in this case the mean. Each factor is plotted and all mean levels of the factor (two or more) are connected with straight lines. The plot is meant to complement other analyses, such as the typical analysis of variance. == Example == Shown below is an example from Box, Hunter, and Hunter (1978). In it, several factors are chosen to determine their effect(s) on bike speed. These factors are: seat height, dynamo engaged (yes/no), angle of the handlebar, chosen gear, raincoat (on/off), breakfast (yes/no), and tire pressure (low/high), each at two levels. The plot shows the mean response level for each variable (vertical) versus the factor (horizontal). To interpret the graph, all that is needed is to determine the ordering of the factors by length of the line connecting the two dots. The longer the line, the more significant the factor. Therefore, this plot clearly demonstrates that factor 4 (chosen gear) is


id: double-mass-analysis-20260505-0
file: data/en.wikipedia.org/wiki/Double_mass_analysis-0.md
category: reference
title: Double mass analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Double_mass_analysis
date_saved: 2026-05-05T09:54:07.787564+00:00
instance: kb-cron
parent: double-mass-analysis-20260505-0
summary:
Double mass analysis is a simple graphical method to evaluate the consistency of hydrological data. The DM approach plots the cumulative data of one variable against the cumulative data of a second variable. A break in the slope of a linear function fit to the data is thought to represent a change in the relation between the variables. This approach provides a robust method to determine a change in the behavior of precipitation and recharge in a simple graphical method. It is a commonly used data analysis approach for investigating the behaviour of records made of hydrological or meteorological data at a number of locations. It is used to determine whether there is a need for corrections to the data - to account for changes in data collection procedures or other local conditions. Such changes may result from a variety of things including changes in instrumentation, changes in observation procedures, or changes in gauge location or surrounding conditions. Double mass analysis for checking consistency of a hydrological or meteorological record is considered to be an essential tool before taking it for analysis purpose. This method is based on the hypothesis that each item of the recorded data of a population


id: error-level-analysis-20260505-0
file: data/en.wikipedia.org/wiki/Error_level_analysis-0.md
category: reference
title: Error level analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Error_level_analysis
date_saved: 2026-05-05T09:54:08.944694+00:00
instance: kb-cron
parent: error-level-analysis-20260505-0
summary:
Error level analysis (ELA) is the analysis of compression artifacts in digital data with lossy compression such as JPEG. == Principles == When used, lossy compression is normally applied uniformly to a set of data, such as an image, resulting in a uniform level of compression artifacts. Alternatively, the data may consist of parts with different levels of compression artifacts. This difference may arise from the different parts having been repeatedly subjected to the same lossy compression a different number of times, or the different parts having been subjected to different kinds of lossy compression. A difference in the level of compression artifacts in different parts of the data may therefore indicate that the data has been edited. In the case of JPEG, even a composite with parts subjected to matching compressions will have a difference in the compression artifacts. In order to make the typically faint compression artifacts more readily visible, the data to be analyzed is subjected to an additional round of lossy compression, this time at a known, uniform level, and the result is subtracted from the original data under investigation. The resulting difference image is then inspected manually for any variation in the level of compression


id: facet-theory-20260505-0
file: data/en.wikipedia.org/wiki/Facet_theory-0.md
category: reference
title: Facet theory
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Facet_theory
date_saved: 2026-05-05T09:54:10.375224+00:00
instance: kb-cron
parent: facet-theory-20260505-0
summary:
Facet theory is a metatheory for the multivariate behavioral sciences that posits that scientific theories and measurements can be advanced by discovering relationships between conceptual classifications of research variables and empirical partitions of data-representation spaces. For this purpose, facet theory proposes procedures for (1) Constructing or selecting variables for observation, using the mapping sentence technique (a formal definitional framework for a system of observations), and (2) Analyzing multivariate data, using data representation spaces, notably those depicting similarity measures (e.g., correlations), or partially ordered sets, derived from the data. Facet theory is characterized by its direct concern with the entire content-universe under study, containing many, possibly infinitely many, variables. Observed variables are regarded just as a sample of statistical units from the multitude of variables that make up the investigated attribute (the content-universe). Hence, Facet theory proposes techniques for sampling variables for observation from the entire content universe; and for making inferences from the sample of observed variables to the entire content universe. The sampling of variables is done with the aid of the mapping sentence technique (see Section 1); and inferences from the sample of observed variables to the entire content universe are made with respect to correspondences between conceptual


id: facet-theory-20260505-1
file: data/en.wikipedia.org/wiki/Facet_theory-1.md
category: reference
title: Facet theory
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Facet_theory
date_saved: 2026-05-05T09:54:10.375224+00:00
instance: kb-cron
parent: facet-theory-20260505-0
summary:
An initial framework for observing intelligence could be Mapping Sentence 1. The mapping sentence serves as a unified semantic device for specifying the system of intelligence test items, according to the present conceptualization. Its content facet, the material facet, may now serve as a classification of intelligence test items to be considered. Thus, in designing observations, a stratified sampling of items is afforded by ensuring an appropriate selection of items from each of the material facet elements; that is, from each class of items: the verbal, the numerical and the spatial. === Enriching the mapping sentence === The research design can be enriched by introducing to the mapping sentence an additional, independent classification of the observations in the form of an additional content-facet, thereby facilitating systematic differentiations of the observations. For example, intelligence items may be classified also according to the cognitive operation required in order to respond correctly to an item: whether rule-recall (memory), rule-application, or rule-inference. Instead of the three sub-content-universes of intelligence defined by the material facet alone, we now have nine sub-content-universes defined by the cartesian multiplication of the material and the mental-operation facets. See mapping sentence 2. Another way of enriching a mapping sentence (and


id: facet-theory-20260505-2
file: data/en.wikipedia.org/wiki/Facet_theory-2.md
category: reference
title: Facet theory
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Facet_theory
date_saved: 2026-05-05T09:54:10.375224+00:00
instance: kb-cron
parent: facet-theory-20260505-0
summary:
Receiving as input (or computing from input data) a matrix of similarity coefficients, specifying, for each pair of items how similar they are. A common example is the computation of a correlation-coefficient matrix from input data, where the size of a correlation coefficient between two variables reflects the degree of similarity between them. Mapping the items (variables) as points in a geometric space of a given dimensionality while preserving as well as possible the condition: If rij>rkl then dij<dkl for all i,j,k,l where rij is the similarity measure (e.g., correlation coefficient) between variables i,j and dij is the distance between their points in the space. Most often, Euclidean distance function (Minkowsky distance of order 2) is used. But other distance functions, especially the Manhattan distance function (Minkowsky distance of order 1) are called for. (See Subsection Relating POSAC Measurement Space to the SSA Concept Space below.) The goodness-of-fit of the resulting mapping may be assessed by a loss function– Kruskal's Stress coefficient or Guttman's Coefficient of Alienation. Partitioning the space as well as possible, into simple regions (stripe, sectors or concentric rings) whose variables are in 1-1 correspondence with a pre-conceived content-facet. To run this option, content facet(s) must be


id: facet-theory-20260505-3
file: data/en.wikipedia.org/wiki/Facet_theory-3.md
category: reference
title: Facet theory
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Facet_theory
date_saved: 2026-05-05T09:54:10.375224+00:00
instance: kb-cron
parent: facet-theory-20260505-0
summary:
=== Principles of faceted SSA: A summary === 1. The attribute under study is represented by a geometric space. 2. Variables of the attribute are represented as points in that space. Conversely, every point in the geometric space is a variable of the attribute. This is the Continuity Principle. 3. The observed variables, located as points in the empirical Faceted SSA map, constitute but a sample drawn from the many (possibly infinitely many) variables constituting the content universe of the attribute investigated. 4. The observed variables chosen for SSA must all belong to the same content universe. This is ensured by including in the SSA only variables whose ranges are similarly ordered with respect to a common meaning (CMR). 5. The sample of variables marked on the Faceted SSA map is used as a guide for inferring possible partitions of the SSA-attribute-map into distinct regions, each region representing a component, or subdomain, of the attribute. 6. In Facet Theory, relationships between attribute components (such as verbal intelligence and numeric intelligence as components of intelligence), are expressed in geometric terms –such as shapes and spatial orientation – rather than in algebraic terms. Just as one would describe relationships between neighboring countries


id: facet-theory-20260505-4
file: data/en.wikipedia.org/wiki/Facet_theory-4.md
category: reference
title: Facet theory
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Facet_theory
date_saved: 2026-05-05T09:54:10.375224+00:00
instance: kb-cron
parent: facet-theory-20260505-0
summary:
Variables processed by Facet Theory measurement operations to be described below, evenly cover the attribute content universe. To ensure such coverage, Facet Theory measurement operations are often performed not on the sample of the observed items themselves, but rather on composite variables that represent facet elements that had been validated by Faceted SSA. The sample of individuals is rich enough to allow existing score-profiles of the processed variables to be observed. In the resulting measurement, order relations among individuals should preserve sufficiently well order relations (including comparability and incomparability; see below) between individuals' profiles of the processed variables. The result of the measurement operation yields the smallest number of scales; The resultant scales represent fundamental variables whose interpretation derives from the contents of the observed items, but does not depend on the particular sample of items observed. Partial order analysis of observed data. Let observed items v1,...,vn with a common-meaning range (CMR) represent an investigated content universe; let A1,...,An be their ranges with each Aj ordered from high to low with respect to the common meaning; and let A = A1×A2 × ... × An be the cartesian product of all the range facets, Aj (j = 1,...,n). A system


id: facet-theory-20260505-5
file: data/en.wikipedia.org/wiki/Facet_theory-5.md
category: reference
title: Facet theory
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Facet_theory
date_saved: 2026-05-05T09:54:10.375224+00:00
instance: kb-cron
parent: facet-theory-20260505-0
summary:
Given this POSAC solution, an attempt is made to interpret the two coordinates, X1 and X2, as two fundamental scales of the investigated phenomenon of evening TV watching by the investigated population. This is done by, first, interpreting the intervals (equivalence classes) within each coordinate, and then trying to conceptualize the derived meanings of the ordered intervals, in terms of a meaningful notion that may be attributed to the coordinate. In the present simplified example, this is easy: Inspecting the map, we attempt to identify the feature that distinguishes all profiles with given score in X1. Thus, we find that profiles with X1=4, and only they, represent TV watching in the fourth hour. Profiles with X1 = 3 all have 1 in the third watching hour but 0 at the fourth hour, i.e., the third hour is the latest watching hour. X1 = 2 is assigned to, and only to, profiles whose latest watching hour is the second hour. And, finally, X1 = 1 is for the profile 1000 which represents the fact that the first hour is the only – and therefore the latest – watching hour (ignoring the profile 0000 of those who didn't watch TV at the


id: facet-theory-20260505-6
file: data/en.wikipedia.org/wiki/Facet_theory-6.md
category: reference
title: Facet theory
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Facet_theory
date_saved: 2026-05-05T09:54:10.375224+00:00
instance: kb-cron
parent: facet-theory-20260505-0
summary:
Interval 1. Low Utility & Low Equality DJ Attitude Interval 2. Low Utility & High Equality DJ Attitude Interval 3. High Utility s & Low Corrective Action DJ Attitude Interval 4. High Utility & High Corrective Action DJ Attitude That is, Enhanced Utility Attitude, even if low, (interval 1 and 2) is somewhat present when Equality is favored (interval 2). If Enhanced Utility Attitude is high (intervals 3 and 4), it reaches the extreme level (interval 4) when Corrective Action is favored. (This may well reflect the sentiment that, in the long run, the advancement of disadvantaged pupils serves the common good.) The meanings of the fundamental variables, X and Y, while relying on the concepts of fairness and of utility, respectively, suggest new notions that modify them. The new notions were christened Enhanced (or Extended) Fairness and Enhanced (or Extended) Utility. === Complementary topics in partial order spaces === Higher order partition lines. The above simple measurement space illustrates partition-lines that are straight or have one bend. More complex measurement spaces result with items whose partition-lines have two or more bends. While partial order spaces are used mainly for analyzing score profiles (based on range facets), under certain conditions,


id: facet-theory-20260505-7
file: data/en.wikipedia.org/wiki/Facet_theory-7.md
category: reference
title: Facet theory
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Facet_theory
date_saved: 2026-05-05T09:54:10.375224+00:00
instance: kb-cron
parent: facet-theory-20260505-0
summary:
== Facet theory: comparisons and comments == Concerned with the entire cycle of multivariate research – concept definition, observational design, and data analysis for concept-structure and measurement, Facet Theory constitutes a novel paradigm for the behavioral sciences. Hence, only limited aspects of it can be compared with specific statistical methods. A distinctive feature of Facet Theory is its explicit concern with the entire set of variables included in the investigated content-universe, regarding the subset of observed variables as but a sample from which inferences can be made. Hence, clusters of variables, if observed, are of no significance. They are simply unimportant artifacts of the procedure for sampling of the variables. This is in contrast with cluster analysis or factor analysis where recorded clustering patterns determine research results and interpretations. There have been various attempts to describe technical differences between Factor Analysis and Facet Theory. Briefly, it may be said that while Factor Analysis aims to structure the set of variables selected for observation, Facet Theory aims to structure the entire content universe of all variables, observed as well as unobserved, relying on the continuity principle and using regional hypotheses as an inferential procedure. Guttman's SSA, as well as Multidimensional Scaling


id: feature-engineering-20260505-0
file: data/en.wikipedia.org/wiki/Feature_engineering-0.md
category: reference
title: Feature engineering
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Feature_engineering
date_saved: 2026-05-05T09:54:11.527727+00:00
instance: kb-cron
parent: feature-engineering-20260505-0
summary:
Feature engineering is a preprocessing step in supervised machine learning and statistical modeling which transforms raw data into a more effective set of inputs. Each input comprises several attributes, known as features. By providing models with relevant information, feature engineering significantly enhances their predictive accuracy and decision-making capability. Beyond machine learning, the principles of feature engineering are applied in various scientific fields, including physics. For example, physicists construct dimensionless numbers such as the Reynolds number in fluid dynamics, the Nusselt number in heat transfer, and the Archimedes number in sedimentation. They also develop first approximations of solutions, such as analytical solutions for the strength of materials in mechanics. == Clustering == One of the applications of feature engineering has been clustering of feature-objects or sample-objects in a dataset. Especially, feature engineering based on matrix decomposition has been extensively used for data clustering under non-negativity constraints on the feature coefficients. These include Non-Negative Matrix Factorization (NMF), Non-Negative Matrix-Tri Factorization (NMTF), Non-Negative Tensor Decomposition/Factorization (NTF/NTD), etc. The non-negativity constraints on coefficients of the feature vectors mined by the above-stated algorithms yields a part-based representation, and different factor matrices exhibit natural clustering properties. Several extensions of the above-stated feature engineering methods have been


id: geographic-analytics-20260505-0
file: data/en.wikipedia.org/wiki/Geographic_analytics-0.md
category: reference
title: Geographic analytics
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Geographic_analytics
date_saved: 2026-05-05T09:54:12.686727+00:00
instance: kb-cron
parent: geographic-analytics-20260505-0
summary:
Geographic analytics is an analytical approach to strategic management and data analytics to make geographic decisions efficiently. Examples of such decisions are choosing the location for a warehouse or planning the regions for a marketing campaign. Data, information and framing conditions are visualized on maps to derive recommendations for action. In comparison to geographic information systems (GIS), which primarily aim at the representation of information on maps (descriptive analytics), Geographic analytics additionally focuses on making business decisions based on the data visualization on the map (prescriptive analytics). == Background == Purely mathematically based approaches that are used in data analytics in order to support management decisions often have the disadvantage that they require a large amount of data. This often results in considerable efforts for gathering, cleaning and understanding the data. Furthermore, there can be “intangible” framing conditions that are disruptive to any purely data-driven optimization solution. Example logistics: For the task to find the optimal location for a warehouse in a logistics network, so-called Center of Gravity models are being used. To minimize the cost, these models use transport volume data, customer locations, cost data, etc., in order to determine the optimal location. However, there are often framing conditions


id: health-care-analytics-20260505-0
file: data/en.wikipedia.org/wiki/Health_care_analytics-0.md
category: reference
title: Health care analytics
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Health_care_analytics
date_saved: 2026-05-05T09:54:13.909352+00:00
instance: kb-cron
parent: health-care-analytics-20260505-0
summary:
Health care analytics is the health care analysis activities that can be undertaken as a result of data collected from four areas within healthcare: (1) claims and cost data, (2) pharmaceutical and research and development (R&D) data, (3) clinical data (such as collected from electronic medical records (EHRs)), and (4) patient behaviors and preferences data (e.g. patient satisfaction or retail purchases, such as data captured in stores selling personal health products). Health care analytics is a growing industry in many countries including the United States, where it is expected to grow to more than $31 billion by 2022. It is also increasingly important to governments and public health agencies to support health policy and meet public expectations for transparency, as accelerated by the COVID-19 pandemic. Health care analytics allows for the examination of patterns in various healthcare data in order to determine how clinical care can be improved for patients and provider teams, while limiting excessive spending and improving the health of populations. Areas of the industry focuses on clinical analysis, financial analysis, supply chain analysis, as well as marketing, fraud and HR analysis. There is increasing demand in many countries to incorporate social indicators of patients and providers within


id: health-care-analytics-20260505-1
file: data/en.wikipedia.org/wiki/Health_care_analytics-1.md
category: reference
title: Health care analytics
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Health_care_analytics
date_saved: 2026-05-05T09:54:13.909352+00:00
instance: kb-cron
parent: health-care-analytics-20260505-0
summary:
=== Federal government role in health IT === In the United States, multiple federal entities are heavily involved in health analytics infrastructure. Within the executive branch, the administration itself, Centers for Medicare and Medicaid Services (CMS), and Office of the National Coordinator for Health Information Technology (ONC) each have strategic plans and are involved in determining regulation. Within the legislative branch, multiple committees within the House of Representatives and Senate hold hearings and have opinions on using data and technology to reduce costs and improve outcomes in healthcare. The ONC issued the Federal Health IT Strategic Plan 2015-2020. The plan outlines the steps federal agencies will take to achieve widespread use of health information technology (health IT) and electronic health information to enhance the health IT infrastructure, to advance person-centered and self-managed health, to transform health care delivery and community health, and to foster research, scientific knowledge and innovation. The plan is intended “to provide clarity in federal policies, programs, and actions and includes strategies to align program requirements, harmonize and simplify regulations, and aims to help health IT users to advance the learning health system to achieve better health.” The Strategic Plan includes several key initiatives employing multiple strategies


id: iconography-of-correlations-20260505-0
file: data/en.wikipedia.org/wiki/Iconography_of_correlations-0.md
category: reference
title: Iconography of correlations
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Iconography_of_correlations
date_saved: 2026-05-05T09:54:15.084817+00:00
instance: kb-cron
parent: iconography-of-correlations-20260505-0
summary:
In exploratory data analysis, the iconography of correlations, or representation of correlations, is a data visualization technique which replaces a numeric correlation matrix by its graphical projection onto a diagram. Correlations between variables are represented by solid lines (positive correlations) or dotted lines (negative correlations) plotted on the diagram, where shorter lengths or thicker lines (or both) represent greater projection components and thus greater degree of correlation. == History == The idea of using graphical models to represent correlations has a notable application in genomic mapping, though the iconography of correlations is more general for not assuming the probability distribution, since it only relies on representing the correlation coefficients geometrically. The iconography of correlations first dates to 1975, applied to marine geochemistry in a 1981 thesis, and later in a 1982 data analysis article. Afterwards, the method was applied widely in the aerospace industry but for about fifteen years manufacturers kept it fairly confidential; generally, they preferred not to broadcast useful techniques to their competitors. In 1997 the first company was incorporated to distribute iconography of correlations software. Since then the topic of iconography of correlations has been incorporated into university courses, and typical topical articles' citation lists have rapidly


id: interleaving-distance-20260505-0
file: data/en.wikipedia.org/wiki/Interleaving_distance-0.md
category: reference
title: Interleaving distance
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Interleaving_distance
date_saved: 2026-05-05T09:54:16.289352+00:00
instance: kb-cron
parent: interleaving-distance-20260505-0
summary:
In topological data analysis, the interleaving distance is a measure of similarity between persistence modules, a common object of study in topological data analysis and persistent homology. The interleaving distance was first introduced by Frédéric Chazal et al. in 2009. since then, it and its generalizations have been a central consideration in the study of applied algebraic topology and topological data analysis. == Definition == A persistence module V {\displaystyle \mathbb {V} } is a collection ( V t ∣ t ∈ R ) {\displaystyle (V_{t}\mid t\in \mathbb {R} )} of vector spaces indexed over the real line, along with a collection ( v t s : V s → V t ∣ s ≤ t ) {\displaystyle (v_{t}^{s}:V_{s}\to V_{t}\mid s\leq t)} of linear maps such that v t t {\displaystyle v_{t}^{t}} is always an isomorphism, and the relation v t s ∘ v s r = v t r {\displaystyle v_{t}^{s}\circ v_{s}^{r}=v_{t}^{r}} is satisfied for every r ≤ s ≤ t {\displaystyle r\leq s\leq t} . The case of R {\displaystyle \mathbb {R} } indexing is presented here for simplicity, though the interleaving distance can be readily adapted to more general settings, including multi-dimensional persistence modules. Let U {\displaystyle


id: item-tree-analysis-20260505-0
file: data/en.wikipedia.org/wiki/Item_tree_analysis-0.md
category: reference
title: Item tree analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Item_tree_analysis
date_saved: 2026-05-05T09:54:17.467755+00:00
instance: kb-cron
parent: item-tree-analysis-20260505-0
summary:
Item tree analysis (ITA) is a data analytical method which allows constructing a hierarchical structure on the items of a questionnaire or test from observed response patterns. Assume that we have a questionnaire with m items and that subjects can answer positive (1) or negative (0) to each of these items, i.e. the items are dichotomous. If n subjects answer the items this results in a binary data matrix D with m columns and n rows. Typical examples of this data format are test items which can be solved (1) or failed (0) by subjects. Other typical examples are questionnaires where the items are statements to which subjects can agree (1) or disagree (0). Depending on the content of the items it is possible that the response of a subject to an item j determines her or his responses to other items. It is, for example, possible that each subject who agrees to item j will also agree to item i. In this case we say that item j implies item i (short i → j {\displaystyle i\rightarrow j} ). The goal of an ITA is to uncover such deterministic implications from the data set D. == Algorithms for ITA


id: item-tree-analysis-20260505-1
file: data/en.wikipedia.org/wiki/Item_tree_analysis-1.md
category: reference
title: Item tree analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Item_tree_analysis
date_saved: 2026-05-05T09:54:17.467755+00:00
instance: kb-cron
parent: item-tree-analysis-20260505-0
summary:
== Notes == == References == Bart, W. M., & Krus, D. J. (1973). An ordering-theoretic method to determine hierarchies among items. Educational and psychological measurement, 33, 291–300. Duquenne V (1987). Conceptual Implications Between Attributes and some Representation Properties for Finite Lattices. In B Ganter, R Wille, K Wolfe (eds.), Beiträge zur Begriffsanalyse: Vorträge der Arbeitstagung Begriffsanalyse, Darmstadt 1986, pp. 313–339. Wissenschafts-Verlag, Mannheim. Flament C (1976). L’Analyse Bool´eenne de Questionnaire. Mouton, Paris. Held, T., & Korossy, K. (1998). Data-analysis as heuristic for establishing theoretically founded item structures. Zeitschrift für Psychologie, 206, 169–188. Janssens, R. (1999). A Boolean approach to the measurement of group processes and attitudes. The concept of integration as an example. Mathematical Social Sciences, 38, 275–293. Schrepp M (1999). On the Empirical Construction of Implications on Bi-valued Test Items. Mathematical Social Sciences, 38(3), 361–375. Schrepp, M (2002). Explorative analysis of empirical data by boolean analysis of questionnaires. Zeitschrift für Psychologie, 210/2, S. 99-109. Schrepp, M. (2003). A method for the analysis of hierarchical dependencies between items of a questionnaire. Methods of Psychological Research, 19, 43–79. Schrepp, M. (2006). ITA 2.0: A program for Classical and Inductive Item Tree Analysis. Journal of Statistical Software, Vol. 16, Issue 10. Schrepp,


id: key-value-database-20260505-0
file: data/en.wikipedia.org/wiki/Key–value_database-0.md
category: reference
title: Key–value database
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Key–value_database
date_saved: 2026-05-05T09:54:18.590764+00:00
instance: kb-cron
parent: key-value-database-20260505-0
summary:
A key-value database, or key-value store, is a data storage paradigm designed for storing, retrieving, and managing associative arrays, a data structure more commonly known today as a dictionary. Dictionaries contain a collection of objects, or records, which in turn have many different fields within them. These records are stored and retrieved using a key that uniquely identifies the record, and is used to find the data within the database. Key-value databases differ from the better known relational databases (RDB). RDBs pre-define the data structure in the database as a series of tables containing fields with well-defined data types. Exposing the data types to the database program allows it to apply various optimizations. In contrast, key-value systems treat the value as opaque to the database itself, and typically support only simple operations such as storing, retrieving, updating, and deleting a value by its key. This offers considerable flexibility and makes such systems well suited to low-latency, high-throughput workloads dominated by direct key lookups, but less suitable for applications that require complex queries or explicit relationships among records. A lack of standardization, limited transaction support, and relatively simple query interfaces long restricted many key-value systems to specialized uses, but the rapid


id: l1-norm-principal-component-analysis-20260505-0
file: data/en.wikipedia.org/wiki/L1-norm_principal_component_analysis-0.md
category: reference
title: L1-norm principal component analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/L1-norm_principal_component_analysis
date_saved: 2026-05-05T09:54:19.890827+00:00
instance: kb-cron
parent: l1-norm-principal-component-analysis-20260505-0
summary:
L1-norm principal component analysis (L1-PCA) is a general method for multivariate data analysis. L1-PCA is often preferred over standard L2-norm principal component analysis (PCA) when the analyzed data may contain outliers (faulty values or corruptions), as it is believed to be robust. Both L1-PCA and standard PCA seek a collection of orthogonal directions (principal components) that define a subspace wherein data representation is maximized according to the selected criterion. Standard PCA quantifies data representation as the aggregate of the L2-norm of the data point projections into the subspace, or equivalently the aggregate Euclidean distance of the original points from their subspace-projected representations. L1-PCA uses instead the aggregate of the L1-norm of the data point projections into the subspace. In PCA and L1-PCA, the number of principal components (PCs) is lower than the rank of the analyzed matrix, which coincides with the dimensionality of the space defined by the original data points. Therefore, PCA or L1-PCA are commonly employed for dimensionality reduction for the purpose of data denoising or compression. Among the advantages of standard PCA that contributed to its high popularity are low-cost computational implementation by means of singular-value decomposition (SVD) and statistical optimality when the data set is generated


id: l1-norm-principal-component-analysis-20260505-1
file: data/en.wikipedia.org/wiki/L1-norm_principal_component_analysis-1.md
category: reference
title: L1-norm principal component analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/L1-norm_principal_component_analysis
date_saved: 2026-05-05T09:54:19.890827+00:00
instance: kb-cron
parent: l1-norm-principal-component-analysis-20260505-0
summary:
The transition from (5) to (8) for K = 1 {\displaystyle K=1} holds true, since the unique singular value of X b {\displaystyle \mathbf {X} \mathbf {b} } is equal to ‖ X b ‖ 2 = b ⊤ X ⊤ X b {\displaystyle \|\mathbf {X} \mathbf {b} \|_{2}={\sqrt {\mathbf {b} ^{\top }\mathbf {X} ^{\top }\mathbf {X} \mathbf {b} }}} , for every b {\displaystyle \mathbf {b} } . Then, if b BNM {\displaystyle \mathbf {b} _{\text{BNM}}} is the solution to BQM in (7), it holds that is the exact L1-PC of X {\displaystyle \mathbf {X} } , as defined in (1). In addition, it holds that b BNM = sgn ( X ⊤ q L1 ) {\displaystyle \mathbf {b} _{\text{BNM}}={\text{sgn}}(\mathbf {X} ^{\top }\mathbf {q} _{\text{L1}})} and ‖ X ⊤ q L1 ‖ 1 = ‖ X b BNM ‖ 2 {\displaystyle \|\mathbf {X} ^{\top }\mathbf {q} _{\text{L1}}\|_{1}=\|\mathbf {X} \mathbf {b} _{\text{BNM}}\|_{2}} . == Algorithms == === Exact solution of exponential complexity === As shown above, the exact solution to L1-PCA can be obtained by the following two-step process: 1. Solve the problem in (5) to obtain B BNM {\displaystyle \mathbf {B} _{\text{BNM}}} . 2. Apply SVD on X B


id: lesion-network-mapping-20260505-0
file: data/en.wikipedia.org/wiki/Lesion_network_mapping-0.md
category: reference
title: Lesion network mapping
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Lesion_network_mapping
date_saved: 2026-05-05T09:54:21.110794+00:00
instance: kb-cron
parent: lesion-network-mapping-20260505-0
summary:
Lesion network mapping is a neuroimaging technique that analyzes the connectivity pattern of brain lesions to identify neuroanatomic correlates of symptoms. The scientific validity of these methods has been widely disputed. The technique was developed by Aaron Boes to understand the network anatomy of lesion induced neurologic and psychiatric symptoms that can not be explained by focal anatomic localization. Lesion network mapping applies a network-based approach to identify connected brain networks, rather than focal brain regions, that correlate with a specific symptom. In focal neuroanatomic localization, developed by Paul Broca and others, specific symptoms that occur due to brain lesions can be understood by identifying a specific brain region that is injured by lesions to establish brain-symptom relationships. However, a number of neurologic symptoms, such as peduncular hallucinosis, are not amenable to this approach since the lesions associated with the symptom do not map to one focal brain location. Lesion network mapping helps to explain these lesion-induced syndromes by showing that lesion locations associated with a given symptom all map to a shared brain network even if they do not all map to a focal brain region. The technique maps the location of lesions associated with a specific symptom and


id: mathematical-biosciences-institute-20260505-0
file: data/en.wikipedia.org/wiki/Mathematical_Biosciences_Institute-0.md
category: reference
title: Mathematical Biosciences Institute
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Mathematical_Biosciences_Institute
date_saved: 2026-05-05T09:54:22.360776+00:00
instance: kb-cron
parent: mathematical-biosciences-institute-20260505-0
summary:
The Mathematical Biosciences Institute (MBI) was an institution of higher learning affiliated with the Ohio State University in Columbus, Ohio. MBI received major funding from the National Science Foundation. == History == Under the leadership of founding director Avner Friedman, MBI opened in September 2002, holding its first workshop, hosting its first visiting researchers, and starting its first cohort of postdocs in that month. MBI holds 10–12 scientific workshops each year, and hosts about 25 postdoctoral and visiting researchers in residence at any given time. Through its collective events and programs, MBI draws over 1000 visits by researchers in the broadly defined area of mathematical biology throughout the year. MBI's long term planning is overseen by its Directorate and its Board of Trustees, while its scientific activities are overseen by its Directorate and its Scientific Advisory Committee. == MBI programs == === Workshops === MBI organizes Emphasis Semesters consisting of three or four week-long workshops. Emphasis Semesters are organized around selected themes which have included Mathematical Neuroscience, Cancer and Its Environment, and Analysis of Complex Data in Biological Systems. Outside of Emphasis Semesters, Current Topic Workshops focus on emerging topics in the mathematical biosciences. Most of the institute's programs are


id: meter-data-management-20260505-0
file: data/en.wikipedia.org/wiki/Meter_data_management-0.md
category: reference
title: Meter data management
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Meter_data_management
date_saved: 2026-05-05T09:54:23.521041+00:00
instance: kb-cron
parent: meter-data-management-20260505-0
summary:
Meter data management (MDM) refers to software that performs long-term data storage and management for the vast quantities of data delivered by smart metering systems. This data consists primarily of usage data and events that are imported from the head-end servers managing the data collection in advanced metering infrastructure (AMI) or automatic meter reading (AMR) systems. MDM is a component in the smart grid infrastructure promoted by utility companies. This may also incorporate meter data analytics, the analysis of data emitted by electric smart meters that record consumption of electric energy. == MDM systems == An MDM system will typically import the data, then validate, cleanse and process it before making it available for billing and analysis. Products for meter data include: Smart meter deployment planning and management; Meter and network asset monitoring and management; Automated smart meter provisioning (i.e. addition, deletion and updating of meter information at utility and AMR side) and billing cutover; Meter-to-cash system, workforce management system, asset management and other systems. Furthermore, an MDM may provide reporting capabilities for load and demand forecasting, management reports, and customer service metrics. An MDM provide application programming interfaces (APIs) between the MDM and the multiple destinations that rely on


id: multi-model-database-20260505-0
file: data/en.wikipedia.org/wiki/Multi-model_database-0.md
category: reference
title: Multi-model database
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Multi-model_database
date_saved: 2026-05-05T09:54:24.674447+00:00
instance: kb-cron
parent: multi-model-database-20260505-0
summary:
In the field of database design, a multi-model database is a database management system designed to support multiple data models against a single, integrated backend. In contrast, most database management systems are organized around a single data model that determines how data can be organized, stored, and manipulated. Document, graph, relational, and key–value models are examples of data models that may be supported by a multi-model database. == Background == The relational data model became popular after its publication by Edgar F. Codd in 1970. Due to increasing requirements for horizontal scalability and fault tolerance, NoSQL databases became prominent after 2009. NoSQL databases use a variety of data models, with document, graph, and key–value models being popular. A multi-model database is a database that can store, index and query data in more than one model. For some time, databases have primarily supported only one model, such as: relational database, document-oriented database, graph database or triplestore. A database that combines many of these is multi-model. This should not be confused with multimodal database systems such as Pixeltable or ApertureDB, which focus on unified management of different media types (images, video, audio, text) rather than different data models. For some time, it


id: multivariate-logistic-regression-20260505-0
file: data/en.wikipedia.org/wiki/Multivariate_logistic_regression-0.md
category: reference
title: Multivariate logistic regression
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Multivariate_logistic_regression
date_saved: 2026-05-05T09:54:25.883559+00:00
instance: kb-cron
parent: multivariate-logistic-regression-20260505-0
summary:
Multivariate logistic regression is a type of data analysis that predicts any number of outcomes based on multiple independent variables. It is based on the assumption that the natural logarithm of the odds has a linear relationship with independent variables. == Procedure == First, the baseline odds of a specific outcome compared to not having that outcome are calculated, giving a constant (intercept). Next, the independent variables are incorporated into the model, giving a regression coefficient (beta) and a "P" value for each independent variable. The "P" value determines how significantly the independent variable impacts the odds of having the outcome or not. It is desirable to use as few variables as necessary, and to have at least 10 - 20 times as many observations as independent variables. === Formula === Multivariate logistic regression uses a formula similar to univariate logistic regression, but with multiple independent variables. where v is the number of independent variables. The following formula shows that multivariate logistic regression is simply a standard linear regression model: == Types == The two main types of multivariate logistic regression are linear regression and logistic regression. === Linear regression === Linear regression produces results that show a linear relationship


id: multiway-data-analysis-20260505-0
file: data/en.wikipedia.org/wiki/Multiway_data_analysis-0.md
category: reference
title: Multiway data analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Multiway_data_analysis
date_saved: 2026-05-05T09:54:27.058609+00:00
instance: kb-cron
parent: multiway-data-analysis-20260505-0
summary:
Multiway data analysis is a method of analyzing large data sets by representing a collection of observations as a multiway array, A ∈ C I 0 × I 1 × … I c × … I C {\displaystyle {\mathcal {A}}\in {\mathbb {C} }^{I_{0}\times I_{1}\times \dots I_{c}\times \dots I_{C}}} . The proper choice of data organization into (C+1)-way array, and analysis techniques can reveal patterns in the underlying data undetected by other methods. == History == The study of multiway data analysis was first formalized as the result of a conference held in 1988. The result of this conference was the first text specifically addressed to this field, Coppi and Bolasco's Multiway Data Analysis. At that time, the application areas for multiway analysis included statistics, econometrics and psychometrics. In recent years, applications have expanded to include chemometrics, agriculture, social network analysis and the food industry. == Composition of multiway data analysis == === Multiway data === Multiway data analysts use the term way to refer to the number sources of data variation while reserving the word mode for the methods or models used to analyze the data. In this sense, we can define the various ways of data to analyze: One


id: narrative-intelligence-20260505-0
file: data/en.wikipedia.org/wiki/Narrative_intelligence-0.md
category: reference
title: Narrative intelligence
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Narrative_intelligence
date_saved: 2026-05-05T09:54:28.284976+00:00
instance: kb-cron
parent: narrative-intelligence-20260505-0
summary:
Narrative Intelligence refers to the capability of identifying, understanding and responding to narrative attacks that cause financial reputational, operational or physical harm. It is studied at the intersection of artificial intelligence (AI), cybersecurity, linguistics, psychology, and cognitive science. The term is used to describe how computational systems and humans structure, interpret, and apply stories in communication, reasoning, and decision-making. == Definition and scope == Narrative intelligence encompasses three primary areas: story identification, story understanding, and narrative response. Story identification refers to the capacity to identify harmful narratives by their structural components, including characters, events, settings, goals, and causal relationships. In computational research, this involves extracting structured representations from natural language texts in order to model the underlying narrative structure. Story understanding concerns knowing the context of coherent and contextually consistent narratives by computational systems. Approaches to story understanding include rule-based systems, planning-based models, and AI-based learning techniques. These systems aim to produce narratives that maintain logical consistency across events and adhere to temporal and causal constraints. Narrative response involves drawing inferences from narrative content and then making decisions on how to respond to them. This includes modeling causal and temporal relationships, predicting possible outcomes, and supporting decision-making processes based on


id: necessary-condition-analysis-20260505-0
file: data/en.wikipedia.org/wiki/Necessary_condition_analysis-0.md
category: reference
title: Necessary condition analysis
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Necessary_condition_analysis
date_saved: 2026-05-05T09:54:29.420634+00:00
instance: kb-cron
parent: necessary-condition-analysis-20260505-0
summary:
Necessary condition analysis (NCA) is a research approach and tool employed to discern "necessary conditions" within datasets. These indispensable conditions stand as pivotal determinants of particular outcomes, wherein the absence of such conditions ensures the absence of the intended result. For example, the admission of a student into a Ph.D. program necessitates a prior degree; the progression of AIDS necessitates the presence of HIV; and organizational change necessitates communication. The absence these conditions guarantees the outcome cannot occur, and no other condition can overcome the lack of this condition. Further, necessary conditions are not always sufficient. For example, AIDS necessitates HIV, but HIV does not always cause AIDS. In such instances, the condition demonstrates its necessity but lacks sufficiency. NCA seeks to use statistical methods to test for such conditions. == Overview == Traditional statistical methods often emphasize the identification of factors that are sufficient to produce an outcome. In contrast, NCA aims to uncover conditions that must be present for a specific outcome to occur. While researchers will sometimes use NCA as a stand-alone analysis, they often use it to add additional depth to existing analyses of data. For example, NCA acts as stand-alone method or as a complement


id: negative-testing-20260505-0
file: data/en.wikipedia.org/wiki/Negative_testing-0.md
category: reference
title: Negative testing
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Negative_testing
date_saved: 2026-05-05T09:54:30.657459+00:00
instance: kb-cron
parent: negative-testing-20260505-0
summary:
Negative testing is a method of testing an application or system to improve the likelihood that an application works as intended/specified and can handle unexpected input and user behavior. Invalid data is inserted to compare the output against the given input. Negative testing is also known as failure testing or error path testing. When performing negative testing exceptions are expected. This shows that the application is able to handle improper user behavior. Users input values that do not work in the system to test its ability to handle incorrect values or system failure. == Purpose == The purpose of negative testing is to prevent the application from crashing and it also helps improve the quality of an application by detecting defects. Negative testing helps you to improve the testing coverage of the application. Negative testing makes the application more stable and reliable. Negative testing together with positive testing allows users to test the application with any valid (or invalid) input data. == Benefits of negative testing == Negative testing is done to check that the product deals properly with the circumstance for which it is not programmed. The fundamental aim of this testing is to check how bad data is


id: netvibes-20260505-0
file: data/en.wikipedia.org/wiki/Netvibes-0.md
category: reference
title: Netvibes
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Netvibes
date_saved: 2026-05-05T09:54:31.800507+00:00
instance: kb-cron
parent: netvibes-20260505-0
summary:
Netvibes is a French brand of Dassault Systèmes that previously ran a web service offering a dashboard and feed reader. Currently, the company offers business intelligence tools. == History == === 2005–2012 === Founded in 2005 by Tariq Krim, the company provided software for personalized dashboards for real-time monitoring, social analytics, knowledge sharing, and decision support. === 2012–present === On February 9, 2012, Dassault Systèmes announced the acquisition of Netvibes. As of 2024, Netvibes also contains the operations of two other software companies acquired by Dassault Systèmes: Exalead: founded in 2000 by François Bourdoncle, the company provided search platforms and search-based applications for consumer and business users. On June 9, 2010, Dassault Systèmes acquired the company. Proxem: Founded in 2007 by François-Régis Caumartin, the company provided AI-powered semantic processing software and services. On June 23, 2020, Dassault Systèmes acquired Proxem and integrated its technology into the 3DEXPERIENCE® platform to complement its information intelligence applications. Dassault Systèmes announced in April 2025 that Netvibes would retire its standalone web service offering on June 2, 2025. == Activities == Brand monitoring – to track clients, customers and competitors across media sources all in one place, analyze live results with third party reporting tools,


id: nosql-20260505-0
file: data/en.wikipedia.org/wiki/NoSQL-0.md
category: reference
title: NoSQL
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/NoSQL
date_saved: 2026-05-05T09:54:32.988294+00:00
instance: kb-cron
parent: nosql-20260505-0
summary:
NoSQL (originally meaning "not only SQL" or "non-relational") refers to a type of database design that stores and retrieves data differently from the traditional table-based structure of relational databases. Unlike relational databases, which organize data into rows and columns like a spreadsheet, NoSQL databases use a single data structure—such as key–value pairs, wide columns, graphs, or documents—to hold information. Since this non-relational design does not require a fixed schema, it scales easily to manage large, often unstructured datasets. NoSQL systems are sometimes called "Not only SQL" because they can support SQL-like query languages or work alongside SQL databases in polyglot-persistent setups, where multiple database types are combined. Non-relational databases date back to the late 1960s, but the term "NoSQL" emerged in the early 2000s, spurred by the needs of Web 2.0 companies like social media platforms. NoSQL databases are popular in big data and real-time web applications due to their simple design, ability to scale across clusters of machines (called horizontal scaling), and precise control over data availability. These structures can speed up certain tasks and are often considered more adaptable than fixed database tables. However, many NoSQL systems prioritize speed and availability over strict consistency (per the CAP theorem),


id: nosql-20260505-1
file: data/en.wikipedia.org/wiki/NoSQL-1.md
category: reference
title: NoSQL
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/NoSQL
date_saved: 2026-05-05T09:54:32.988294+00:00
instance: kb-cron
parent: nosql-20260505-0
summary:
=== Multiple queries === Instead of retrieving all the data with one query, it is common to do several queries to get the desired data. NoSQL queries are often faster than traditional SQL queries, so the cost of additional queries may be acceptable. If an excessive number of queries would be necessary, one of the other two approaches is more appropriate. === Caching, replication and non-normalized data === Instead of only storing foreign keys, it is common to store actual foreign values along with the model's data. For example, each blog comment might include the username in addition to a user id, thus providing easy access to the username without requiring another lookup. When a username changes, however, this will now need to be changed in many places in the database. Thus this approach works better when reads are much more common than writes. === Nesting data === With document databases like MongoDB it is common to put more data in a smaller number of collections. For example, in a blogging application, one might choose to store comments within the blog post document, so that with a single retrieval one gets all the comments. Thus in this approach a single


id: offset-filtration-20260505-0
file: data/en.wikipedia.org/wiki/Offset_filtration-0.md
category: reference
title: Offset filtration
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Offset_filtration
date_saved: 2026-05-05T09:54:34.194182+00:00
instance: kb-cron
parent: offset-filtration-20260505-0
summary:
The offset filtration (also called the "union-of-balls" or "union-of-disks" filtration) is a growing sequence of metric balls used to detect the size and scale of topological features of a data set. The offset filtration commonly arises in persistent homology and the field of topological data analysis. Utilizing a union of balls to approximate the shape of geometric objects was first suggested by Frosini in 1992 in the context of submanifolds of Euclidean space. The construction was independently explored by Robins in 1998, and expanded to considering the collection of offsets indexed over a series of increasing scale parameters (i.e., a growing sequence of balls), in order to observe the stability of topological features with respect to attractors. Homological persistence as introduced in these papers by Frosini and Robins was subsequently formalized by Edelsbrunner et al. in their seminal 2002 paper Topological Persistence and Simplification. Since then, the offset filtration has become a primary example in the study of computational topology and data analysis. == Definition == Let X {\displaystyle X} be a finite set in a metric space ( M , d ) {\displaystyle (M,d)} , and for any x ∈ X {\displaystyle x\in X} let B ( x ,


id: open-coding-20260505-0
file: data/en.wikipedia.org/wiki/Open_coding-0.md
category: reference
title: Open coding
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Open_coding
date_saved: 2026-05-05T09:54:35.377999+00:00
instance: kb-cron
parent: open-coding-20260505-0
summary:
Based in grounded theory, open coding is the analytic process through which concepts (codes) are attached to observed data and phenomena during qualitative data analysis. It is one of the techniques described by Strauss (1987) and Strauss and Corbin (1990) for working with text. Open coding attempts to codify, name or classifying the observed phenomenon and is achieved by segmenting data into meaningful expressions and describing that data with a single word or short sequence of words. Relevant annotations and concepts are then attached to these expressions. == Details == Applied in varying degrees of detail, open coding can be linked to a line, sentence, paragraph or complete text (e.g., protocol, scenario). Alternatives are selected according to the research question, relevant data, personal style of the analyst and the stage of research. However, coding should always follow its aim to break down and understand a text and develop a scheme of categories over the course of time. The result of open coding should be a list of characteristic codes and categories attached to the text and supported by code notes to explain the content. These notes could take the form of interesting observations and thoughts that are relevant to the


id: open-data-index-20260505-0
file: data/en.wikipedia.org/wiki/Open_data_index-0.md
category: reference
title: Open data index
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/Open_data_index
date_saved: 2026-05-05T09:54:36.535575+00:00
instance: kb-cron
parent: open-data-index-20260505-0
summary:
An open data index is any set of indicators which assesses and evaluates the general openness of a government's open data portal. Open data indices not only show how open a data portal is, but also encourage citizens and government officials alike, to participate in their local open data communities, particularly in advocating for local open data and local open data policies. There are two mainstream methodologies, which are Global Open Data Index and Open Data Barometer. The Global Open Data Index evaluates an open data portal from 11 different aspects based on the Open Definition of open data, while the Open Data Barometer adds two more indices compared to the previous one. == Scoring standard == According to the service offered by Open Knowledge International, they run a measurement called "Global Open Data Index" which is "an annual effort to measure the state of open government data around the world". And they evaluate the openness of an open dataset according to the following questions: 1. Does the data exist? (5 marks) The Open Knowledge Foundation specifically indicates that the data of an open data portal should be directly comes from the official government department or a third party with


id: opendap-20260505-0
file: data/en.wikipedia.org/wiki/OPeNDAP-0.md
category: reference
title: OPeNDAP
tags: science, encyclopedia
source_url: https://en.wikipedia.org/wiki/OPeNDAP
date_saved: 2026-05-05T09:54:37.688368+00:00
instance: kb-cron
parent: opendap-20260505-0
summary:
OPeNDAP (Open-source Project for a Network Data Access Protocol) is an endeavor focused on enhancing the retrieval of remote, structured data through a Web-based architecture and a discipline-neutral Data Access Protocol (DAP). == Project == Widely used, especially in Earth science, the protocol is layered on HTTP, and its current specification is DAP4, though the previous DAP2 version remains broadly used. Developed and advanced (openly and collaboratively) by the non-profit OPeNDAP, Inc., DAP is intended to enable remote, selective data-retrieval as an easily invoked Web service. OPeNDAP, Inc. also develops and maintains zero-cost (reference) implementations of the DAP protocol in both server-side and client-side software. "OPeNDAP" often is used in place of "DAP" to denote the protocol but also may refer to an entire DAP-based data-retrieval architecture. Other DAP-centered architectures, such as THREDDS and ERDDAP, the NOAA GEO-IDE UAF ERDDAP exhibit significant interoperability with one another as well as with systems employing OPeNDAP's own (open-source) servers and software. A DAP client can be an ordinary browser or even a spreadsheet, though with limited functionality (see OPeNDAP's Web page on Available Client Software). More typically, DAP clients are: Data-analysis or data-visualization tools (such as MATLAB, IDL, Panoply, GrADS, Integrated Data Viewer,


