The bounce rate of a website is simply the number of visitors that leave after only viewing one page; essentially the number of visitors who bounce away from the site before delving deeper.
Bounce rate can greatly affect your search engine ranking. A search engine's top priority is to provide its users with the best possible results, so it is constantly churning data to achieve this.
One of the things that is measured is the amount of time a person spends on a website before bouncing back to the search engine for another option; if this is constantly a very short amount of time, 5 seconds or less, the search engine concludes that the site is poor and so lowers its ranking in the results. SEO requires more than people visiting your site, it needs them to like it, talk about it and keep coming back so the longer your potential clients spend on it the better for SEO. Reducing your bounce rate is as simple as having a top quality website that people don't want to click away from.
So, if bounce rate is the number of people who never venture onto any other page on a website, then exit rate is the number of people who left your website from a particular page. You will have an exit rate for each section of the site. The people who left your site from a certain page may have navigated there via your 'contact us' or 'about us' page, but found themselves at the top and left from there. A high exit rate isn't necessarily a bad thing as users may have read through your whole site backwards before leaving from the main content, but if people are constantly exiting your site before the final page, then you may have some navigation issues to consider.