Statistics is a
mathematical science pertaining to the collection, analysis, interpretation or
explanation and presentation of data. It provides tools for predicting and
forecasting the economic, marketing, and other activities.
Meaning of Statistics: Statistics is concerned with scientific methods for
collecting, organizing, summarizing, presenting, and analyzing data as well as
deriving valid conclusions and making reasonable decisions on the basis of this
analysis. Statistics is concerned with the systematic collection of numerical
data and its interpretation. The word ‘statistic’ is used to refer to 1.
Numerical facts, such as the number of people living in particular area. 2. The
study of ways of collecting, analyzing and interpreting the facts.
TYPES OF STATISTICS:
1. Descriptive Statistics consists of methods for organizing, displaying, and
describing data by using tables, graphs, and summary measures.
2. Inferential Statistics consists of methods that use sample results to help make decisions
or predictions about a population.
DESCRIPTIVE STATISTICS
Descriptive statistics is the term given to the analysis of
data that helps describe, show or summarize data in a meaningful way such that,
for example, patterns might emerge from the data. Descriptive statistics do
not, however, allow us to make conclusions beyond the data we have analyzed or
reach conclusions regarding any hypotheses we might have made. They are simply
a way to describe our data.
Descriptive statistics are very important because if we
simply presented our raw data it would be hard to visualize what the data was
showing, especially if there was a lot of it. Descriptive statistics therefore
enables us to present the data in a more meaningful way, which allows simpler
interpretation of the data. For example, if we had the results of 100
pieces of students' coursework, we may be interested in the overall performance
of those students. We would also be interested in the distribution or spread of
the marks. Descriptive statistics allow us to do this.
Typically, there are five general types of statistics that
are used to describe data:
A. Frequency table In
statistics, a frequency distribution is a list, table or graph that displays
the frequency of various outcomes in a sample. Each entry in the table contains
the frequency or count of the occurrences of values within a particular group
or interval. B. Measures of central tendency (Mean, Median &
Mode)Measures of central tendency: these
are ways of describing the central position of a frequency distribution for a
group of data. In this case, the frequency distribution is simply the
distribution and pattern of marks scored by the 100 students from the lowest to
the highest. We can describe this central position using a number of statistics,
including mode, median and mean. Read more about Measures of Central Tendency by clicking the
link Measures of central tendency
C. Measures of spread or Dispersion: Variance, Range Measures of spread or Dispersion: these are ways of summarizing a group of data by describing
how spread out the scores is. For example, the mean score of our 100 students
may be 65 out of 100. However, not all students will have scored 65 marks.
Rather, their scores will be spread out. Some will be lower and others higher.
Measures of spread help us to summarize how spread out these scores are. To
describe this spread, a number of statistics are available to us, including the
range, quartiles, absolute deviation, variance and standard deviation.
When we use descriptive statistics
it is useful to summarize our group of data using a combination of tabulated
description (i.e., tables), graphical description (i.e., graphs and charts) and
statistical commentary (i.e., a discussion of the results).
Measures of spread describe how similar or varied the set of observed values
are for a particular variable (data item). Measures of spread include:1. Range 2. variance 3. Standard Deviation Read more about Range, Variance and Standard Deviation by clicking the links: Range Mean Deviation Standard Deviation D. Distribution - Skewness and Kurtosis The distribution is a mathematical function that describes the
relationship of observations of different heights. A distribution is simply
a collection of data, or scores, on a variable. Usually, these scores are
arranged in order from smallest to largest and then they can be presented
graphically.
a. Skewness - Skewness is a measure
of symmetry, or more precisely, the lack of symmetry. A distribution, or data
set, is symmetric if it looks the same to the left and right of the center
point. Skewness characterizes the degree of asymmetry of a distribution around
its mean.
The
concept of skewness is mainly used to understand the distribution of the
data and steps taken to normalize the data for further building of machine learning models. In
case of negatively skewed data, Mean<Median<Mode. This indicates, more
data points are to the right of the curve here the data has very high values in
large numbers.
b. Kurtosis - Kurtosis
characterizes the symmetric distribution through the relative peaks or flatness
of the curve. Kurtosis is a measure of whether the data are heavy-tailed or
light-tailed relative to a normal distribution.
The
main difference between Skewness and Kurtosis is: Skewness
measures the degree of the slope in the frequency distribution. Kurtosis
measures the degree of thickness in the tails of the distribution curve.
There are 3 types of Kurtosis: • Platykurtic (relatively flat) • Mesokurtic (normal) • Leptokurtic (relatively peaked)
E. Measures of Position ( Percentile, Quartile ) What is Percentile? Percentiles
indicate the percentage of scores that fall below a particular value. They tell
you where a score stands relative to other scores. For example, a person with
an IQ of 120 is at the 91st percentile, which indicates that their IQ is higher
than 91 percent of other scores.
Percentiles are a great tool to use when you need to know the relative
standing of a value. Where does a value fall within a distribution of values?
While the concept behind percentiles is straight forward, there are different mathematical
methods for calculating them. In this post, learn about percentiles, special
percentiles and their surprisingly flexible uses, and the various procedures
for calculating them.
Using
Percentiles to Understand a Score or Value - Percentiles
tell you how a value compares to other values. The general rule is that if
value X is at the kth percentile, then X is greater than K% of the values. What
Is a Quartile? A quartile is a statistical term that describes a
division of observations into four defined intervals based on the values of the
data and how they compare to the entire set of observations.
Understanding
Quartiles To understand the quartile, it is important to understand the median as
a measure of central tendency. The median in statistics is the middle value of
a set of numbers. It is the point at which exactly half of the data lies below
and above the central value. So, given a set of 13 numbers, the median would be
the seventh number. The six numbers preceding this value are the lowest numbers
in the data, and the six numbers after the median are the highest numbers in
the dataset given. Because the median is not affected by extreme values or
outliers in the distribution, it is sometimes preferred to the mean. The median
is a robust estimator of location but says nothing about how the data on either
side of its value is spread or dispersed. That's where the quartile steps in.
The quartile measures the spread of values above and below the mean by dividing
the distribution into four groups. How
Quartiles Work Just like the median divides the data into half so
that 50% of the measurement lies below the median and 50% lies above it, the
quartile breaks down the data into quarters so that 25% of the measurements are
less than the lower quartile, 50% are less than the median, and 75% are less
than the upper quartile.
A quartile
divides data into three points—a lower quartile, median, and upper quartile—to
form four groups of the dataset. The lower quartile, or first quartile, is
denoted as Q1 and is the middle number that falls between the smallest value of
the dataset and the median. The second quartile, Q2, is also the median. The
upper or third quartile, denoted as Q3, is the central point that lies between
the median and the highest number distribution.
Now,
we can map out the four groups formed from the quartiles. The first group of
values contains the smallest number up to Q1; the second group includes Q1 to
the median; the third set is the median to Q3; the fourth category comprises Q3
to the highest data point of the entire set. Each quartile contains 25% of the total observations. Generally,
the data is arranged from smallest to largest: 1. First quartile: the lowest 25% of numbers 2. Second quartile: between 25.1% and 50% (up to the
median) 3. Third quartile: 50.1% to 75% (above the median) 4. Fourth
quartile: the highest 25% of numbers
INFERENTIAL STATISTICS Inferential statistics are the statistical procedures
that are used to reach conclusions about associations between variables. They
differ from descriptive statistics in that they are explicitly designed
to test hypotheses.
Inferential statistics is mainly used to derive estimates about a
large group (or population) and draw conclusions on the data based on
hypotheses testing methods. Inferential statistics makes use of sample data because it is more
cost-effective and less tedious than collecting data from an entire population.
It allows one to come to reasonable assumptions about the larger population
based on a sample’s characteristics. Sampling methods need to be unbiased and
random for statistical conclusions and inferences to be validated. Inferential statistics is strongly associated with the logic of
hypothesis testing.
With inferential statistics, you are trying to reach conclusions
that extend beyond the immediate data alone. For instance, we use inferential
statistics to try to infer from the sample data what the population might
think.
Or, we use inferential statistics to make judgments of the probability
that an observed difference between groups is a dependable one or one that
might have happened by chance in this study. Thus, we use inferential
statistics to make inferences from our data to more general conditions; we use
descriptive statistics simply to describe what’s going on in our data. Inferential statistics are useful in experimental and
quasi-experimental research design or in program outcome evaluation.
Perhaps one
of the simplest inferential test is used when you want to compare the average
performance of two groups on a single measure to see if there is a difference.
You might want to know whether eighth-grade boys and girls differ in math test
scores or whether a program group differs on the outcome measure from a control
group. Whenever you wish to compare the average performance between two groups
you should consider the t-test for differences between groups. Most of the major inferential statistics come from a general family of statistical models known as the General Linear Model. This includes the t-test, Analysis of Variance (ANOVA), Analysis of Covariance (ANCOVA), regression analysis, and many of the multivariate methods like factor analysis, multidimensional scaling, cluster analysis, discriminant function analysis, and so on. Given the importance of the General Linear Model, it’s a good idea for any serious social researcher to become familiar with its workings. The discussion of the General Linear Model here is very elementary and only considers the simplest straight-line model. However, it will get you familiar with the idea of the linear model and help prepare you for the more complex analyses described below.
Thus, Inferential statistics helps to suggest explanations for a situation or phenomenon. It allows you to draw conclusions based on extrapolations and is in that way fundamentally different from descriptive statistics that merely summarize the data that has actually been measured.
The most common methodologies in inferential statistics are hypothesis tests, confidence intervals, and regression analysis. Interestingly, these inferential methods can produce similar summary values as descriptive statistics, such as the mean and standard deviation. The following types of inferential statistics are extensively used and relatively easy to interpret:
1. Pearson Correlation
2. Regression
3. T-test
4. Factor Analysis
5. Disciminant analysis
CORRELATION AND LINEAR REGRESSION The most commonly used techniques for investigating the relationship between two quantitative variables are correlation and linear regression.
• Correlation quantifies the strength of the linear relationship between a pair of variables
• Correlation is a single statistic, or data point
• Correlation shows the relationship between the two variables
• Regression expresses the relationship in the form of an equation.
• Regression is the entire equation with all of the data points that are represented with a line.
• Regression allows us to see how one affects the other
It all comes down to correlation and regression, which are statistical analysis measurements used to find connections between two variables, measure the connections, and make predictions. Measuring correlation and regression is commonly used in a variety of industries, and it can also be seen in our daily lives.
For instance, have you ever seen someone driving an expensive car and automatically thought that the driver must be financially successful? Or how about thinking that the further you run on your morning workout, the more weight you’ll lose?
Both of these are examples of real-life correlation and regression, as you’re seeing one variable (a fancy car or a long workout) and then seeing if there is any direct relation to another variable (being wealthy or losing weight). As we investigate the relationships between two variables, it’s important to know the differences and the similarities between correlation and regression.
Correlation vs. regression - Its not uncommon for correlation and regression to be confused for one another as correlation can often drive into regression. However, there is a key difference.
What is the difference between correlation and regression?
The difference between these two statistical measurements is that correlation measures the degree of a relationship between two variables (x and y), whereas regression is how one variable affects another.
Basically, you need to know when to use correlation vs. regression. Use correlation for a quick and simple summary of the direction and strength of the relationship between two or more numeric variables. Use regression when you’re looking to predict, optimize, or explain a number response between the variables (how x influences y).
What is correlation? When
it comes to correlation, think of it as the combination of the words “co”
meaning together and “relation” meaning a connection between two quantities. In
this sense, correlation is when a change to one variable is then followed by a
change in another variable, whether it is direct or indirect. Variables are
considered “uncorrelated” when a change in one does not affect the other. In
short, it measures the relationship between two variables. For
example, let’s say our two variables are x and y. The changes
between these two variables can be considered positive or negative. A positive
change would be when two variables move in the same direction, meaning an
increase in one variable result in an increase in another variable. So, if an
increase in x increases y, it’s positively correlated. An
example of this would be demand and price. This is because an increase in
demand causes the corresponding increase in price. The price would increase
because there are more consumers who want it are willing to pay more for it. If
two variables are moving in opposite directions, like when an increase in one
variable results in a decrease in another, this is known as a negative
correlation. An example of a negative correlation would be the price and demand
for a product because an increase in price (x) results in a decrease in
demand (y). Knowing
how two variables are correlated allows for predicting trends in the future, as
you’ll be able to understand the relationship between the variables — or if
there's no relationship at all.
Correlation analysis The main purpose of correlation, through the lens of correlation analysis, is to allow experimenters to know the association or the absence of a relationship between two variables. When these variables are correlated, you’ll be able to measure the strength of their association. Overall, the objective of correlation analysis is to find the numerical value that shows the relationship between the two variables and how they move together. One key benefit of correlation is that it is a more concise and clear summary of the relationship between the two variables than you’ll find with regression.
Example of correlation A correlation chart, also known as a scatter diagram, makes it
easier to visually see the correlation between two variables. Data in a
correlation chart is represented by a single point. In the chart above you can
see that correlation plots various points of single data. Let's think of
correlation as real-life scenarios. In addition to the price and demand
example, let's take a look at correlation from a marketing standpoint to see
the strength of a relationship between the two variables. For instance, it
could be in your company's best interest to see if there is a predictable
relationship between the sale of a product and factors like weather,
advertising, and consumer income.
What is regression? On the other hand, regression is how one variable affects another,
or changes in a variable that trigger changes in another, essentially cause and
effect. It implies that the outcome is dependent on one or more variables. For instance, while correlation can be defined as the relationship between two variables, regression is how they affect each other. An example of this would be how an increase in rainfall would then cause various crops to grow, just like a drought would cause crops to wither or not grow at all.
Regression analysis Regression analysis helps to determine the functional relationship between two variables (x and y) so that you’re able to estimate the unknown variable to make future projections on events and goals.
The main objective of regression analysis is to estimate the values of a random variable (z) based on the values of your known (or fixed) variables (x and y). Linear regression analysis is considered to be the best fitting line through the data points. Let’s use the example of tracking the value of a single share in the stock market over the years. X will be time in years and Y will be the value in dollars. We know that the value of a stock is changed by time passing, among other things. We can’t control those other things, but we can control when we sell the stock, so we control the time variable. But how dependent is the value of a stock on time passed? If we bought a stock for $1 and in one year its value went up to $100, does that mean every year the value will go up another $100? Does that mean in 25 years it will be valued at $2500? We don’t know. We figure it out by looking at how much the stock earned over several years. That’s fairly simple because we’re only measuring how much we change one thing or one variable. Then we put those measurements on a graph or plot. The dots could be all over the place or scattered. Could we draw a line through the dots that would show a trend? Let’s call that a trendline. Yes, we can certainly try. That line is a simple linear regression trendline through a scatter plot. For e.g. given below if the data of increase in share price year after year for 20 years. Time (Yrs) and stock value: 2000 - 1498, 2001 - 1160, 2002 - 1147, 2003 - 848.
The main advantage in using regression within your analysis is
that it provides you with a detailed look of your data (more detailed than
correlation alone) and includes an equation that can be used for predicting and
optimizing your data in the future. When the line is drawn using regression, we can see two pieces of
information: Regression
formula A → refers to the y-intercept, the value of y when x = 0 B
→ refers to the slope, or rise over run The prediction formula used to see how data could look in the
future is: Y = a + b(x)
Differences between correlation and regression
There are some key differences between correlation and regression
that are important in understanding the two.
• Regression
establishes how x causes y to change, and the results will change
if x and y are swapped. With correlation, x and y are
variables that can be interchanged and get the same result.
• Correlation is a single statistic, or data point, whereas
regression is the entire equation with all of the data points that are
represented with a line. • Correlation shows the relationship between the two variables,
while regression allows us to see how one affects the other.
• The data shown with regression establishes a cause and effect,
when one changes, so does the other, and not always in the same direction. With
correlation, the variables move together.
Similarities between correlation and regression
In addition to differences, there are some key similarities
between correlation and regression that can help you to better understand your
data. • Both work to quantify the direction and strength of the
relationship between two numeric variables.
• Any time the correlation is negative, the regression slope (line
within the graph) will also be negative.
• Any time the correlation is positive, the regression slope (line
within the graph) will be positive. So much more than just cause and effect
•
Even though they’re studied together, it’s clear that there are
obvious differences and similarities between correlation and regression. When
you’re looking to build a model, an equation, or predict a key response, use
regression. If you’re looking to quickly summarize the direction and strength
of a relationship, correlation is your best bet.
T-Test
A t-test is a type of inferential statistic used to determine if
there is a significant difference between the means of two groups, which may be
related in certain features. It is mostly used when the data sets, like the
data set recorded as the outcome from flipping a coin 100 times, would follow a
normal distribution and may have unknown variances. A t-test is used as a
hypothesis testing tool, which allows testing of an assumption applicable to a
population.
For example:
•
• Compare if the people of one country are taller than people of
another one.
•
• Compare if the brain of a person is more activated while
watching happy movies than sad movies.
This comparison can be analyzed by conducting different
statistical analysis, such as t-test
Factor analysis:
Factor analysis is a way to condense the data in many variables
into a just a few variables. For this reason, it is also sometimes called
“dimension reduction.” You can reduce the “dimensions” of your data into one or
more “super-variables.” The most common technique is known as Principal
Component Analysis (PCA). Factor analysis is used to uncover the latent
structure of a set of variables. It reduces attribute space from a large
no. of variables to a smaller no. of factors and as such is a non dependent
procedure.
Example.
What
underlying attitudes lead people to respond to the questions on a political
survey as they do? Examining the correlations among the survey items reveals
that there is significant overlap among various subgroups of items--questions
about taxes tend to correlate with each other, questions about military issues
correlate with each other, and so on. With factor analysis, you can investigate
the number of underlying factors and, in many cases, identify what the factors
represent conceptually. Additionally, you can compute factor scores for each
respondent, which can then be used in subsequent analyses. For example, you
might build a logistic regression model to predict voting behavior based on
factor scores.
Discriminant analysis
Discriminant analysis is a versatile statistical method often
used by market researchers to classify observations into two or more groups or
categories. In other words, discriminant analysis is used to assign objects
to one group among a number of known groups. By performing discriminant
analysis, researchers are able to address classification problems in which two
or more groups, clusters, or populations are known up front, and one or more
new observations are placed into one of the known classifications based on
measured characteristics.
Discriminant analysis is also used to investigate how variables
contribute to group separation, and to what degree. For this reason, it’s often
leveraged to compliment the findings of cluster analysis.
Market researchers are continuously faced with situations in which
their goal is to obtain a better understanding of how groups (customers, age
cohorts, etc.) or items (brands, ideas, etc.), differ in terms of a set of
explanatory or independent variables.
These situations are where discriminant analysis serves as a
powerful research and analysis tool.
Ref: Dr. Hanif Lakdawala's AMR Notes
