Why does the naive bayes algorithm make the naive assumption that features are independent to each other?
Why does the naive bayes algorithm make the naive assumption that features are independent to each other?
Naive Bayes is called naive because it makes the naive assumption that features have zero correlation with each other. They are independent of each other. Why does naive Bayes want to make such an assumption?
Naive Bayes
3 Answers
3
By doing so, the joint distribution can be found easily by just multiplying the probability of each feature whilst in the real world they may not be independent and you have to find the correct joint distribution. It is naive due to this simplification.
Generally, naive Bayes is a very simple and is not used most of the time. There are other learning approaches that are better for learning mappings. About the second part, it highly depends on your task.
– Media
Sep 2 at 12:17
Naive bayes make such assumptions to simplify the calculations. You can take a look at bayesian belief network which do not make such assumptions
Just to complete the answers given and clarify them in some points: the assumption in Naïve Bayes is that features are conditionally independent given the predicted variable, not independent. Note also that, even though this simplification makes naïve assumptions about the conditional joint distribution of features that are in many cases far from the true distribution, our aim here is not to estimate probabilities but to perform a binary classification and, for that purpose, our simplification strategy may be good enough.
Thanks for contributing an answer to Data Science Stack Exchange!
But avoid …
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
But avoid …
To learn more, see our tips on writing great answers.
Required, but never shown
Required, but never shown
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
Thanks for answer. Upvoted and selected as answer. Given this naive assumption, is feature engineering particularly important for naive bayes in selecting the right features which are truly independent of each other? Is fewer features better to minimize the possibility for error?
– user781486
Sep 2 at 11:54