21 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Category utility | 3/5 | https://en.wikipedia.org/wiki/Category_utility | reference | science, encyclopedia | 2026-05-05T15:13:02.093903+00:00 | kb-cron |
I
(
F
a
;
C
)
=
∑
v
i
∈
F
a
∑
c
j
∈
C
p
(
v
i
,
c
j
)
log
p
(
v
i
|
c
j
)
p
(
v
i
)
=
∑
v
i
∈
F
a
∑
c
j
∈
C
p
(
v
i
|
c
j
)
p
(
c
j
)
[
log
p
(
v
i
|
c
j
)
−
log
p
(
v
i
)
]
=
∑
v
i
∈
F
a
∑
c
j
∈
C
p
(
v
i
|
c
j
)
p
(
c
j
)
log
p
(
v
i
|
c
j
)
−
∑
v
i
∈
F
a
∑
c
j
∈
C
p
(
v
i
|
c
j
)
p
(
c
j
)
log
p
(
v
i
)
=
∑
v
i
∈
F
a
∑
c
j
∈
C
p
(
v
i
|
c
j
)
p
(
c
j
)
log
p
(
v
i
|
c
j
)
−
∑
v
i
∈
F
a
∑
c
j
∈
C
p
(
v
i
,
c
j
)
log
p
(
v
i
)
=
∑
v
i
∈
F
a
∑
c
j
∈
C
p
(
v
i
|
c
j
)
p
(
c
j
)
log
p
(
v
i
|
c
j
)
−
∑
v
i
∈
F
a
log
p
(
v
i
)
∑
c
j
∈
C
p
(
v
i
,
c
j
)
=
∑
v
i
∈
F
a
∑
c
j
∈
C
p
(
v
i
|
c
j
)
p
(
c
j
)
log
p
(
v
i
|
c
j
)
−
∑
v
i
∈
F
a
p
(
v
i
)
log
p
(
v
i
)
{\displaystyle {\begin{aligned}I(F_{a};C)&=\sum _{v_{i}\in F_{a}}\sum _{c_{j}\in C}p(v_{i},c_{j})\log {\frac {p(v_{i}|c_{j})}{p(v_{i})}}\\&=\sum _{v_{i}\in F_{a}}\sum _{c_{j}\in C}p(v_{i}|c_{j})p(c_{j})\left[\log p(v_{i}|c_{j})-\log p(v_{i})\right]\\&=\sum _{v_{i}\in F_{a}}\sum _{c_{j}\in C}p(v_{i}|c_{j})p(c_{j})\log p(v_{i}|c_{j})-\sum _{v_{i}\in F_{a}}\sum _{c_{j}\in C}p(v_{i}|c_{j})p(c_{j})\log p(v_{i})\\&=\sum _{v_{i}\in F_{a}}\sum _{c_{j}\in C}p(v_{i}|c_{j})p(c_{j})\log p(v_{i}|c_{j})-\sum _{v_{i}\in F_{a}}\sum _{c_{j}\in C}p(v_{i},c_{j})\log p(v_{i})\\&=\sum _{v_{i}\in F_{a}}\sum _{c_{j}\in C}p(v_{i}|c_{j})p(c_{j})\log p(v_{i}|c_{j})-\sum _{v_{i}\in F_{a}}\log p(v_{i})\sum _{c_{j}\in C}p(v_{i},c_{j})\\&=\sum _{v_{i}\in F_{a}}\sum _{c_{j}\in C}p(v_{i}|c_{j})p(c_{j})\log p(v_{i}|c_{j})-\sum _{v_{i}\in F_{a}}p(v_{i})\log p(v_{i})\\\end{aligned}}}
If the original definition of the category utility from above is rewritten with
C
=
{
c
,
c
¯
}
{\displaystyle C=\{c,{\bar {c}}\}}
,
C
U
(
C
,
F
)
=
∑
f
i
∈
F
∑
c
j
∈
C
p
(
f
i
|
c
j
)
p
(
c
j
)
log
p
(
f
i
|
c
j
)
−
∑
f
i
∈
F
p
(
f
i
)
log
p
(
f
i
)
{\displaystyle CU(C,F)=\sum _{f_{i}\in F}\sum _{c_{j}\in C}p(f_{i}|c_{j})p(c_{j})\log p(f_{i}|c_{j})-\sum _{f_{i}\in F}p(f_{i})\log p(f_{i})}
This equation clearly has the same form as the (blue) equation expressing the mutual information between the feature set and the category variable; the difference is that the sum
∑
f
i
∈
F
{\displaystyle \textstyle \sum _{f_{i}\in F}}
in the category utility equation runs over independent binary variables
F
=
{
f
i
}
,
i
=
1
…
n
{\displaystyle F=\{f_{i}\},\ i=1\ldots n}
, whereas the sum
∑
v
i
∈
F
a
{\displaystyle \textstyle \sum _{v_{i}\in F_{a}}}
in the mutual information runs over values of the single
m
n
{\displaystyle m^{n}}
-ary variable
F
a
{\displaystyle F_{a}}
. The two measures are actually equivalent then only when the features
{
f
i
}
{\displaystyle \{f_{i}\}}
, are independent (and assuming that terms in the sum corresponding to
p
(
f
i
¯
)
{\displaystyle p({\bar {f_{i}}})}
are also added).
== Insensitivity of category utility to ordinality ==
Like the mutual information, the category utility is not sensitive to any ordering in the feature or category variable values. That is, as far as the category utility is concerned, the category set {small,medium,large,jumbo} is not qualitatively different from the category set {desk,fish,tree,mop} since the formulation of the category utility does not account for any ordering of the class variable. Similarly, a feature variable adopting values {1,2,3,4,5} is not qualitatively different from a feature variable adopting values {fred,joe,bob,sue,elaine}. As far as the category utility or mutual information are concerned, all category and feature variables are nominal variables. For this reason, category utility does not reflect any gestalt aspects of "category goodness" that might be based on such ordering effects. One possible adjustment for this insensitivity to ordinality is given by the weighting scheme described in the article for mutual information.
== Category "goodness": models and philosophy == This section provides some background on the origins of, and need for, formal measures of "category goodness" such as the category utility, and some of the history that lead to the development of this particular metric.