Working with Binary Data

In text classification, a binary feature can record whether a word appears in a sentence: one value represents presence and the other represents absence.

Programming

Start with the Feature Values

The important question is not simply whether your data is numerical. The important question is what each feature value represents. A feature might record whether something is present, how many times it occurs, or a continuous measurement. Naive Bayes provides different model variants for these different kinds of data: BernoulliNB assumes binary data, MultinomialNB assumes count data, and GaussianNB can be applied to continuous data.

Which Naive Bayes model matches the meaning of each feature value?

Presence Versus Frequency

Binary data records a two-way condition. In text classification, a binary feature can record whether a word appears in a sentence: one value represents presence and the other represents absence. Count data preserves more detail. Instead of recording only whether a word appears, it records how often the word appears as an integer count. The same sentence can therefore produce two different feature representations.

Worked example

One Sentence, Two Representations

Represent the sentence "cat dog cat" using the vocabulary [cat, dog]. First record presence or absence. Then record the number of appearances.

1
Check presence
The word cat appears, and the word dog appears. A presence-based representation records both features as present.
2
Count appearances
The word cat appears twice, while dog appears once. A count-based representation preserves those integer frequencies.

Result

Using the vocabulary [cat, dog], the binary representation is [1, 1], while the count representation is [2, 1].

How does the same sentence change when features record presence instead of frequency?

Selecting the Model

Model	Assumed feature type	Text-data interpretation
BernoulliNB	Binary data	Whether a feature is present or absent
MultinomialNB	Count data	How many times a feature appears
GaussianNB	Continuous data	A continuous feature value

Choose the variant according to what each feature value represents.

For text classification, BernoulliNB and MultinomialNB are the two variants most closely associated with the feature representations described here. Use BernoulliNB when the features express binary presence or absence. Use MultinomialNB when the features express integer counts, such as how often a word appears in a sentence. GaussianNB belongs with continuous data rather than these binary or count representations.

From Sentence to Classifier

A text-classification workflow begins with words in a sentence and ends with features that a model can use. First, decide what the features mean. If each feature answers whether a word appears, the resulting representation is binary and fits BernoulliNB. If each feature records an integer word count, the representation is count data and fits MultinomialNB. The model choice follows the representation; it is not determined merely by the fact that the original input was text.

How does text become the kind of feature data expected by BernoulliNB or MultinomialNB?

Mistakes in Model Selection

Common mistake
Treating every numeric feature as continuous data
What it looks like
A feature value of 2 may mean that a word appeared twice, rather than representing a continuous measurement.
Why it fails
MultinomialNB assumes count data, while GaussianNB is the variant described for continuous data.
Do this instead
Ask whether the number is an integer count of something before choosing the model.
Common mistake
Using BernoulliNB when frequency information matters
What it looks like
The sentence contains cat twice and dog once, but the representation stores only whether each word appeared.
Why it fails
A binary representation does not preserve the difference between one occurrence and multiple occurrences.
Do this instead
Use a count representation and MultinomialNB when the features are intended to record how often words appear.
Common mistake
Assuming text automatically requires one specific Naive Bayes variant
What it looks like
Two text representations can use the same vocabulary while storing either presence values or integer counts.
Why it fails
The source distinguishes BernoulliNB and MultinomialNB by the type of feature data, even though both are mostly used in text data classification.
Do this instead
Select the variant after identifying whether the text features are binary or count-valued.

Check Your Choice

EASY

A text dataset represents each vocabulary word with the number of times it appears in a sentence. Which Naive Bayes variant matches this feature representation, and why?

Hints

Identify whether the features record presence or integer frequency.
Match the feature type to the model assumptions.

Your attempt (not saved)

Worked example

Practice Solution

A text dataset represents each vocabulary word with the number of times it appears in a sentence. Which Naive Bayes variant matches this feature representation?

1
Identify the data type
The features are integer counts because they record how often a word appears.
2
Match the assumption
MultinomialNB assumes count data.

Result

MultinomialNB is the matching variant.

Key Takeaways

1
BernoulliNB assumes binary features that represent two-way conditions such as presence or absence.
2
MultinomialNB assumes count features, such as how often a word appears in a sentence.
3
GaussianNB can be applied to continuous data.
4
BernoulliNB and MultinomialNB are mostly used for text data classification.
5
Choose the model from the meaning of the feature values, not merely from the fact that the input is text.

At a glance

Binary features record presence or absence.
Count features record integer frequencies.
Use BernoulliNB for binary data, MultinomialNB for count data, and GaussianNB for continuous data.
In text classification, the representation chosen for the words determines which Naive Bayes variant fits.

Related concepts

More concepts starting with W