{"id":70767,"date":"2020-09-09T21:08:49","date_gmt":"2020-09-09T21:08:49","guid":{"rendered":"https:\/\/80000hours.org\/?p=70767"},"modified":"2024-09-03T18:34:28","modified_gmt":"2024-09-03T18:34:28","slug":"good-judgement","status":"publish","type":"post","link":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/","title":{"rendered":"Notes on good judgement and how to develop&nbsp;it"},"content":{"rendered":"<div style=\"width: 100%; min-height: 75px; clear: both;\" class=\"margin-top-smaller margin-bottom-smaller\">\n    <type-3-player \n      cover-image-url=\"\"\n      background-color=\"#f1f1f1\"\n      listen-to-this-page=\"\"\n      listen-to-this-page-text=\"\"\n      link-to-timestamp=\"true\"\n      header-play-buttons=\"true\"\n      sticky=\"true\"\n      primary-color=\"#333\"\n      secondary-color=\"#aaa\"\n      accent-color=\"#2ebdd1\"\n      primary-font-family=\"'museo-sans','Helvetica Neue',Helvetica,Arial,sans-serif\"\n      secondary-font-family=\"'proxima-nova',Arial,sans-serif\"\n      analytics=\"custom\"\n      t3a-logo=\"false\"\n      link-to-timestamp-selector=\".type-3-player__replace-timestamps-with-links\"\n      feedback-button=\"false\"\n      custom-css=\" \n    .type-3-player.sticky {\n      box-shadow: 0px -2px 10px 0px rgb(171 171 171 \/ 60%);\n      z-index: 8000;\n      opacity: 1;\n      visibility: visible;\n      transition: opacity 0.2s ease, visibility 0.2s ease;\n    }\n    .type-3-player.sticky.hidden:not(.loading) {\n      display: flex !important;\n      opacity: 0;\n      visibility: hidden;\n    }\n    .-t3a-button-pill span {\n      padding-top: 1px !important;\n      font-weight: 600;\n    }\n  \"\n    ><\/type-3-player><script type=\"text\/javascript\">\n      window.t3aAnalytics = function(eventType, event) {\n        analytics.track(eventType, event);\n        gtag(\"event\", eventType, event);\n        if (typeof plausible === \"function\") { \n          plausible(eventType, {props: event});\n        }\n      }\n      window.addEventListener(\"scroll\", function() {\n        const players = document.querySelectorAll(\"type-3-player\");\n        const tocButton = document.querySelector(\".sidebar-toc__open-button-wrap\");\n        const scrollTop = window.scrollY;\n        const viewportHeight = window.innerHeight;\n        const totalHeight = document.documentElement.scrollHeight;\n        if (scrollTop + viewportHeight >= totalHeight) {\n          players.forEach(player => player.style.display = \"none\");\n          if (tocButton) {\n            tocButton.style.display = \"none\";\n          }\n        } else {\n          players.forEach(player => player.style.display = \"\");\n          if (tocButton) {\n            tocButton.style.display = \"\";\n          }\n        }\n      });\n    <\/script><script type=\"text\/javascript\">\n      document.addEventListener(\"DOMContentLoaded\", function() {\n        const noPlayButtonTitles = [\"Read more\", \"Read next\", \"Learn more\"];\n        document.querySelectorAll(\"h2, h3\").forEach(function(element) {\n          if (noPlayButtonTitles.includes(element.textContent.trim())) {\n            element.classList.add(\"no-heading-play-button\");\n          }\n        });\n      });\n    <\/script><\/div>\n<p>Judgement, which I roughly define as &#8216;the ability to weigh complex information and reach calibrated conclusions,&#8217; is clearly a valuable skill.<\/p>\n<p>In <a href=\"https:\/\/80000hours.org\/articles\/skills-most-employable\/\">our simple analysis<\/a> of which skills make people most employable, using data from the Bureau of Labor Statistics across the US economy, &#8216;judgement and decision making&#8217; came out top (though meant in a broader sense than we do).<\/p>\n<p>My guess is that good judgement is <em>even more<\/em> important when aiming to have a positive impact.<\/p>\n<p>What follows are some notes on why good judgement matters, what it is, and what we know about how to improve it.<\/p>\n<div id=\"toc_container\" class=\"toc_white no_bullets\"><p class=\"toc_title\">Table of Contents<\/p><ul class=\"toc_list\"><li><a href=\"#why-good-judgement-is-so-valuable-when-aiming-to-have-an-impact\"><span class=\"toc_number toc_depth_1\">1<\/span> Why good judgement is so valuable when aiming to have an impact<\/a><\/li><li><a href=\"#more-on-what-good-judgement-is\"><span class=\"toc_number toc_depth_1\">2<\/span> More on what good judgement is<\/a><\/li><li><a href=\"#some-notes-on-how-to-improve-your-judgement\"><span class=\"toc_number toc_depth_1\">3<\/span> Some notes on how to improve your judgement<\/a><ul><li><a href=\"#spend-an-hour-or-two-doing-calibration-training\"><span class=\"toc_number toc_depth_2\">3.1<\/span> Spend an hour or two doing calibration training<\/a><\/li><li><a href=\"#practice-forecasting\"><span class=\"toc_number toc_depth_2\">3.2<\/span> Practice forecasting<\/a><\/li><li><a href=\"#apply-these-techniques\"><span class=\"toc_number toc_depth_2\">3.3<\/span> Apply these techniques<\/a><\/li><li><a href=\"#try-to-develop-the-right-mindset\"><span class=\"toc_number toc_depth_2\">3.4<\/span> Try to develop the right mindset<\/a><\/li><li><a href=\"#spend-time-with-people-who-have-good-judgement\"><span class=\"toc_number toc_depth_2\">3.5<\/span> Spend time with people who have good judgement<\/a><\/li><li><a href=\"#learn-about-whatever-youre-trying-to-make-judgements-about\"><span class=\"toc_number toc_depth_2\">3.6<\/span> Learn about whatever you&#8217;re trying to make judgements about<\/a><\/li><li><a href=\"#learn-about-when-not-to-use-your-judgement\"><span class=\"toc_number toc_depth_2\">3.7<\/span> Learn about when not to use your judgement<\/a><\/li><li><a href=\"#learn-more-about-decision-making-and-good-thinking\"><span class=\"toc_number toc_depth_2\">3.8<\/span> Learn more about decision making and good thinking<\/a><\/li><\/ul><\/li><li><a href=\"#read-next\"><span class=\"toc_number toc_depth_1\">4<\/span> Read next<\/a><ul><li><a href=\"#accurately-predicting-the-future-is-central-to-absolutely-everything\"><span class=\"toc_number toc_depth_2\">4.1<\/span> Accurately predicting the future is central to absolutely everything.<\/a><\/li><li><a href=\"#improving-institutional-decision-making\"><span class=\"toc_number toc_depth_2\">4.2<\/span> Improving institutional decision making<\/a><\/li><li><a href=\"#expected-value-how-can-we-know-what-makes-a-difference-when-uncertain\"><span class=\"toc_number toc_depth_2\">4.3<\/span> Expected value: how can we know what makes a difference when uncertain?<\/a><\/li><\/ul><\/li><\/ul><\/div>\n<h2><span id=\"why-good-judgement-is-so-valuable-when-aiming-to-have-an-impact\" class=\"toc-anchor\"><\/span>Why good judgement is so valuable when aiming to have an impact<\/h2>\n<p>One reason is lack of feedback. We can never be fully certain which issues are most pressing, or which interventions are most effective. Even in an area like global health \u2013 where we have relatively good data on what works \u2013 there has been <a href=\"https:\/\/blog.givewell.org\/2016\/12\/06\/why-i-mostly-believe-in-worms\/\">huge debate<\/a> over the cost effectiveness of even a straightforward intervention like deworming. Deciding whether to focus on deworming requires judgement.<\/p>\n<p>This lack of feedback becomes even more pressing when we come to efforts to reduce existential risks or help the long-term future, and efforts that take a more <a href=\"https:\/\/www.openphilanthropy.org\/blog\/hits-based-giving\">&#8216;hits based&#8217;<\/a> approach to impact. An existential risk can only happen once, so there&#8217;s a limit to how much data we can ever have about what reduces them, and we must mainly rely on judgement.<\/p>\n<p>Reducing existential risks and some of the other areas we focus on are also new fields of research, so we don&#8217;t even have established heuristics or widely accepted knowledge that someone can simply learn and apply in place of using their judgement.<\/p>\n<p>You may not need to make these judgement calls yourself \u2013 but you at least need to have good enough judgement to pick someone else with good judgement to listen to.<\/p>\n<p>In contrast, in other domains it&#8217;s easier to avoid relying on judgement. For instance, in the world of for-profit startups, it&#8217;s possible (somewhat) to try things, gain feedback by seeing what creates revenue, and refine from there. Someone with so-so judgement can use other approaches to pursue a good strategy.<\/p>\n<p>Other fields have other ways of avoiding judgement. In engineering you can use well-established quantitative rules to figure out what works. When you have lots of data, you can use statistical models. Even in more qualitative research like anthropology, there are standard &#8216;best practice&#8217; research methods that people can use. In other areas you can follow traditions and norms that embody centuries of practical experience.<\/p>\n<p>I get the impression that many in effective altruism agree that judgement is a key trait. In the 2020 EA Leaders Forum survey, respondents were asked which traits they would most like to see in new community members over the next five years, and judgement came out highest by a decent margin.<\/p>\n<div id=\"tablepress-174-scroll-wrapper\" class=\"tablepress-scroll-wrapper\">\n<table id=\"tablepress-174\" class=\"tablepress tablepress-id-174 tablepress-responsive\">\n<tbody class=\"row-hover\">\n<tr class=\"row-1 odd\">\n<td class=\"column-1\">Good judgement (weighing complex information and reaching calibrated conclusions)<\/td>\n<td class=\"column-2\">5.8<\/td>\n<\/tr>\n<tr class=\"row-2 even\">\n<td class=\"column-1\">Analytical intelligence<\/td>\n<td class=\"column-2\">5.1<\/td>\n<\/tr>\n<tr class=\"row-3 odd\">\n<td class=\"column-1\">Entrepreneurial mindset (being able to make things happen independently)<\/td>\n<td class=\"column-2\">5.0<\/td>\n<\/tr>\n<tr class=\"row-4 even\">\n<td class=\"column-1\">Independent thinking (developing one's own views)<\/td>\n<td class=\"column-2\">5.0<\/td>\n<\/tr>\n<tr class=\"row-5 odd\">\n<td class=\"column-1\">Altruism\/Prioritizing the common good<\/td>\n<td class=\"column-2\">4.6<\/td>\n<\/tr>\n<tr class=\"row-6 even\">\n<td class=\"column-1\">Honesty\/Transparency<\/td>\n<td class=\"column-2\">4.4<\/td>\n<\/tr>\n<tr class=\"row-7 odd\">\n<td class=\"column-1\">Emotional intelligence\/Social skills<\/td>\n<td class=\"column-2\">4.3<\/td>\n<\/tr>\n<tr class=\"row-8 even\">\n<td class=\"column-1\">Grit and work ethic<\/td>\n<td class=\"column-2\">3.6<\/td>\n<\/tr>\n<tr class=\"row-9 odd\">\n<td class=\"column-1\">Ambition<\/td>\n<td class=\"column-2\">3.3<\/td>\n<\/tr>\n<tr class=\"row-10 even\">\n<td class=\"column-1\">Creativity<\/td>\n<td class=\"column-2\">3.1<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>It&#8217;s also notable that two of the other most desired traits \u2013 analytical intelligence and independent thinking \u2013 both relate to what we might call &#8216;good thinking&#8217; as well. (Though note that this question was only about &#8216;traits,&#8217; as opposed to skills\/expertise or other characteristics.)<\/p>\n<p>I think this makes sense. Someone with unusually good and trusted judgement can decide what an organisation&#8217;s strategy should be, or make large grants. This is valuable in general, though the community also seems to be especially constrained by people able to do this kind of work right now due to the funding overhang. Many of the bottlenecks faced by the community right now also involve research, and this requires a lot of judgement. When we&#8217;ve looked into the traits required to succeed in our <a href=\"https:\/\/80000hours.org\/articles\/high-impact-careers\/#our-list-of-priority-paths\">priority paths<\/a>, good judgement usually seems very important.<\/p>\n<p>One promising feature of good judgement is that it seems more possible to improve than raw intelligence. So, what \u2013 more practically \u2013 is good judgement, and how can one get it?<\/p>\n<h2><span id=\"more-on-what-good-judgement-is\" class=\"toc-anchor\"><\/span>More on what good judgement is<\/h2>\n<p>I introduced a rough definition above, but there&#8217;s a lot of disagreement about what exactly good judgement is, so it&#8217;s worth saying a little more. Many common definitions seem overly broad, making judgement a central trait almost by definition. For instance, the <em>Cambridge Dictionary<\/em> defines it as:<\/p>\n<blockquote><p>\n  The ability to form valuable opinions and make good decisions\n<\/p><\/blockquote>\n<p>While the US Bureau of Labor Statistics defines it as:<\/p>\n<blockquote><p>\n  Considering the relative costs and benefits of potential actions to choose the most appropriate one\n<\/p><\/blockquote>\n<p>I prefer to focus on the rough narrower definition I introduced at the start (and which was used in the survey I mentioned above), which makes judgement more clearly different from other cognitive traits:<\/p>\n<blockquote><p>\n  The ability to weigh complex information and reach calibrated conclusions\n<\/p><\/blockquote>\n<p>More practically, I think of someone with good judgement as someone able to:<\/p>\n<ol>\n<li>Focus on the right questions<\/li>\n<li>When answering those questions, synthesise many forms of weak evidence using good heuristics, and weigh the evidence appropriately<\/li>\n<li>Be resistant to common cognitive biases by having good habits of thinking<\/li>\n<li>Come to well-calibrated conclusions<\/li>\n<\/ol>\n<p>Owen Cotton-Barratt wrote out <a href=\"https:\/\/forum.effectivealtruism.org\/posts\/xMRpu4nAeGeAXy9ns\/good-judgement-and-its-components\">his understanding of good judgement<\/a>, breaking it into &#8216;understanding&#8217; and &#8216;heuristics.&#8217; His notion is a bit broader than mine.<\/p>\n<p>Here are some closely related concepts:<\/p>\n<ul>\n<li><strong>Keith Stanovich&#8217;s work on &#8216;rationality,&#8217;<\/strong> which seems to be something like someone&#8217;s ability to avoid cognitive biases, and is <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/S0160289616303555\">~0.7 correlated with intelligence<\/a> (so, closely related but not exactly the same)<\/li>\n<li><strong>The cluster of traits (listed later) that make someone a good &#8216;superforecaster&#8217;<\/strong> in Philip Tetlock&#8217;s work (Tetlock also claims that intelligence is only modestly correlated with being a superforecaster)<\/li>\n<\/ul>\n<p>Here are some other concepts in the area, but that seem more different:<\/p>\n<ul>\n<li><strong>Intelligence:<\/strong> I think of this as more like &#8216;processing speed&#8217; \u2013 your ability to make connections, have insights, and solve well-defined problems. Intelligence is an aid in good judgement \u2013 since it lets you make more connections \u2013 but the two seem to come apart. We all know people who are incredibly bright but seem to often make dumb decisions. This could be because they&#8217;re overconfident or biased, despite being smart.<\/li>\n<li><strong>Strategic thinking:<\/strong> Good strategic thinking involves being able to identify top priorities, develop a good plan for working towards those priorities, and improve the plan over time. Good judgement is a great aid to strategy, but a good strategy can also make judgement less necessary (e.g. by creating a good backup plan, you can minimise the risks of your judgement being wrong).<\/li>\n<li><strong>Expertise:<\/strong> Knowledge of the topic is useful all else equal, but Tetlock&#8217;s work (covered more below) shows that many experts don&#8217;t have particularly accurate judgement. <\/li>\n<li><strong>Decision making:<\/strong> Good decision making depends on all of the above: strategy, intelligence, and judgement.<\/li>\n<\/ul>\n<h2><span id=\"some-notes-on-how-to-improve-your-judgement\" class=\"toc-anchor\"><\/span>Some notes on how to improve your judgement<\/h2>\n<p>How to improve judgement is an unsolved problem. The best overview I&#8217;ve found on what&#8217;s currently known is <a href=\"https:\/\/www.openphilanthropy.org\/blog\/efforts-improve-accuracy-our-judgments-and-forecasts\">Open Philanthropy&#8217;s review of the research into methods to improve the judgement of their staff<\/a> by Luke Muelhauser. The following suggestions are aligned with what they conclude.<\/p>\n<p>In particular, the suggestions draw significantly on Phil Tetlock&#8217;s research into how to improve forecasting. This is the single best body of research I&#8217;m aware of in the area of improving judgement.<\/p>\n<p>Tetlock&#8217;s research stands out from other research into improving decision making because:<\/p>\n<ul>\n<li>He developed a way to <em>quantitatively measure<\/em> judgement, by tracking people&#8217;s accuracy at predicting events in current affairs.<\/li>\n<li>This meant he could identify the best forecasters and their habits.<\/li>\n<li>What he learned from this was used to create a training programme, which was then tested in a randomised controlled trial, while most ways to improve decision-making techniques have not been rigorously tested. <\/li>\n<\/ul>\n<p>Tetlock wrote a great popular summary of his research, <em>Superforecasting<\/em>. We have a summary of the book and interview with him on <a href=\"https:\/\/80000hours.org\/podcast\/episodes\/prof-tetlock-predicting-the-future\/\">our podcast<\/a> (followed by a <a href=\"https:\/\/80000hours.org\/podcast\/episodes\/philip-tetlock-forecasting-research\/\">second interview<\/a>).<\/p>\n<p>You can see a <a href=\"https:\/\/aiimpacts.org\/evidence-on-good-forecasting-practices-from-the-good-judgment-project-an-accompanying-blog-post\/\">more thorough review of Tetlock&#8217;s work prepared by AI Impacts<\/a>, with a lot of fascinating data. For instance, the training programme was found to improve accuracy by around 10%, with the effect lasting for several years.<\/p>\n<p>Forecasting isn&#8217;t exactly the same as good judgement, but seems very closely related \u2013 it at least requires&#8221;weighing up complex information and coming to calibrated conclusions&#8221;, though it might require other abilities too. That said, I also take good judgement to include <em>picking the right questions,<\/em> which forecasting doesn&#8217;t cover.<\/p>\n<p>All told, I think there&#8217;s enough overlap that if you improve at forecasting, you&#8217;re likely going to improve your general judgement as well. I don&#8217;t cover other ways to improve judgement as much, because I don&#8217;t think they have as much evidence behind them.<\/p>\n<p>So here are some ways to improve your judgement:<\/p>\n<h3><span id=\"spend-an-hour-or-two-doing-calibration-training\" class=\"toc-anchor\"><\/span>Spend an hour or two doing calibration training<\/h3>\n<p>Being well calibrated is an important input into judgement, and I mention it as part of my short definition of judgement at the start. It means being able to quantify your uncertainty so that if you say you&#8217;re 80% confident in a statement, you&#8217;ll be right four out of five times.<\/p>\n<p>This is important because there&#8217;s a big difference between 20% and 80% confidence, but these could <a href=\"https:\/\/hbr.org\/2018\/07\/if-you-say-something-is-likely-how-likely-do-people-think-it-is\">easily both be called &#8216;likely&#8217; in natural language<\/a>.<\/p>\n<p><a href=\"https:\/\/www.openphilanthropy.org\/blog\/efforts-improve-accuracy-our-judgments-and-forecasts#Calibration_training\">There is evidence to suggest<\/a> that people can improve their calibration in just an hour of training, and there&#8217;s some chance this transfers across domains.<\/p>\n<p>For this reason, Open Philanthropy commissioned a calibration training app, which you can <a href=\"https:\/\/www.clearerthinking.org\/single-post\/2019\/10\/16\/Practice-making-accurate-predictions-with-our-new-tool\">try here<\/a>.<\/p>\n<h3><span id=\"practice-forecasting\" class=\"toc-anchor\"><\/span>Practice forecasting<\/h3>\n<p>As with any skill, the best way to improve is to actually practice. To improve your forecasting, you can practice making forecasts \u2013 especially if you also start to apply some of the techniques covered in the next section while doing it.<\/p>\n<p>Here are some ways to practice:<\/p>\n<ul>\n<li>Get in the habit of making predictions once a year or once a quarter (as does <a href=\"https:\/\/slatestarcodex.com\/2020\/04\/29\/predictions-for-2020\/\">Scott Alexander<\/a>, <a href=\"https:\/\/www.vox.com\/future-perfect\/2020\/1\/13\/21055740\/trump-reelection-biden-nomination-brexit-2020-predictions\">Vox<\/a>, and Open Philanthropy), and tracking your accuracy. You can use <a href=\"https:\/\/predictionbook.com\/\">PredictionBook<\/a> for tracking.<\/li>\n<li>Join a forecasting competition with the <a href=\"https:\/\/www.gjopen.com\/challenges\">Good Judgment Project<\/a>, or on <a href=\"https:\/\/www.metaculus.com\/questions\/\">Metaculus<\/a>.<\/li>\n<li>Take the <a href=\"https:\/\/goodjudgment.com\/resources\/online-training\/\">training course<\/a> we mentioned above.<\/li>\n<\/ul>\n<p>One weakness of the research on forecasting is that it doesn&#8217;t cover how to focus on the right questions in the first place. This is an area of active research briefly covered in our <a href=\"https:\/\/80000hours.org\/podcast\/episodes\/philip-tetlock-forecasting-research\/\">second podcast with Tetlock<\/a>.<\/p>\n<p>Keep in mind that having calibrated overall judgement isn&#8217;t the only habit of thought that matters. Within a team you may want some people who generate creative new ideas and advocate for them even when they&#8217;re probably wrong, or challenge the consensus even when it&#8217;s probably right. That may be easier to do if you&#8217;re overconfident, so there may be a tension between what habits are best for individual judgement and what&#8217;s most helpful when contributing to a group&#8217;s collective judgement.<\/p>\n<h3><span id=\"apply-these-techniques\" class=\"toc-anchor\"><\/span>Apply these techniques<\/h3>\n<p>Luke from Open Philanthropy lists a few techniques for improving judgement that have some backing in the research:<\/p>\n<blockquote>\n<ol>\n<li><strong>Train probabilistic reasoning:<\/strong> In one especially compelling study (<a href=\"http:\/\/journal.sjdm.org\/16\/16511\/jdm16511.html\">Chang et al. 2016<\/a>), a single hour of training in probabilistic reasoning noticeably improved forecasting accuracy. Similar training has improved judgemental accuracy in some earlier studies, and is sometimes included in calibration training.<\/li>\n<li><strong>Incentivise accuracy:<\/strong> In many domains, incentives for accuracy are overwhelmed by stronger incentives for other things, such as incentives for appearing confident, being entertaining, or signalling group loyalty. Some studies suggest that accuracy can be improved merely by providing sufficiently strong incentives for accuracy, such as money or the approval of peers.<\/li>\n<li><strong>Think of alternatives:<\/strong> Some studies suggest that judgemental accuracy can be improved by prompting subjects to consider alternate hypotheses.<\/li>\n<li><strong>Decompose the problem:<\/strong> Another common recommendation is to break each problem into easier-to-estimate sub-problems.<\/li>\n<li><strong>Combine multiple judgements:<\/strong> Often, a weighted (and sometimes &#8216;extremized&#8217;) combination of multiple subjects&#8217; judgements outperforms the judgements of any one person.<\/li>\n<\/ol>\n<\/blockquote>\n<p>Here are Tetlock&#8217;s 10 commandments of forecasting (plus one meta-command), <a href=\"https:\/\/aiimpacts.org\/evidence-on-good-forecasting-practices-from-the-good-judgment-project-an-accompanying-blog-post\/\">as summarised by AI Impacts<\/a>:<\/p>\n<blockquote>\n<ol>\n<li><strong>Triage:<\/strong> Don&#8217;t waste time on questions that are &#8216;clocklike&#8217; (where a rule of thumb can get you pretty close to the correct answer) or &#8216;cloudlike&#8217; (where even fancy models can&#8217;t beat a dart-throwing chimp).<\/li>\n<li><strong>Break seemingly intractable problems into tractable sub-problems:<\/strong> This is how Fermi estimation works. One related piece of advice is &#8220;be wary of accidentally substituting an easy question for a hard one,&#8221; e.g. substituting &#8220;Would Israel be willing to assassinate Yasser Arafat?&#8221; for &#8220;Will at least one of the tests for polonium in Arafat&#8217;s body turn up positive?&#8221;<\/li>\n<li><strong>Strike the right balance between inside and outside views:<\/strong> In particular, <em>first<\/em> anchor with the outside view and <em>then<\/em> adjust using the inside view.<\/li>\n<li><strong>Strike the right balance between under- and overreacting to evidence:<\/strong> &#8220;Superforecasters aren&#8217;t perfect Bayesian predictors but they are much better than most of us.&#8221; Usually do many small updates, but occasionally do big updates when the situation calls for it. Take care not to fall for things that seem like good evidence but aren&#8217;t; remember to think about P(E|H)\/P(E|~H) (<a href=\"https:\/\/en.wikipedia.org\/wiki\/Bayes%27_theorem\">Bayes&#8217; theorem<\/a>); remember to avoid the base-rate fallacy.<\/li>\n<li><strong>Look for the clashing causal forces at work in each problem:<\/strong> This is the &#8216;dragonfly-eye perspective,&#8217; which is where you attempt to do a sort of mental wisdom of the crowds: have tonnes of different causal models and aggregate their judgements. Use &#8216;devil&#8217;s advocate&#8217; reasoning. If you think that P, try hard to convince yourself that not-P. You should find yourself saying &#8220;On the one hand\u2026 on the other hand\u2026 on the third hand\u2026&#8221; a lot.<\/li>\n<li><strong>Strive to distinguish as many degrees of doubt as the problem permits, but no more.<\/strong> <\/li>\n<li><strong>Strike the right balance between under- and overconfidence, between prudence and decisiveness.<\/strong><\/li>\n<li><strong>Look for the errors behind your mistakes, but beware of rearview-mirror hindsight biases.<\/strong><\/li>\n<li><strong>Bring out the best in others and let others bring out the best in you:<\/strong> One pervasive guiding principle is &#8220;Don&#8217;t tell people how to do things; tell them what you want accomplished, and they&#8217;ll surprise you with their ingenuity in doing it.&#8221; The other pervasive guiding principle is &#8220;Cultivate a culture in which people \u2014 even subordinates \u2014 are encouraged to dissent and give counterarguments.&#8221;<\/li>\n<li><strong>Master the error-balancing bicycle:<\/strong> This one should have been called practise, practise, practise. Tetlock says that reading the news and generating probabilities isn&#8217;t enough; you need to actually score your predictions so that you know how wrong you were.<\/li>\n<li><strong>Don&#8217;t treat commandments as commandments:<\/strong> Tetlock&#8217;s point here is simply that you should use your judgement about whether to follow a commandment or not; sometimes they should be overridden.<\/li>\n<\/ol>\n<\/blockquote>\n<h3><span id=\"try-to-develop-the-right-mindset\" class=\"toc-anchor\"><\/span>Try to develop the right mindset<\/h3>\n<p>The people with the best judgement also seem to have a certain mindset. Luke again:<\/p>\n<blockquote><p>\n  According to some of the most compelling studies on forecasting accuracy I&#8217;ve seen, correlates of good forecasting ability include &#8220;thinking like a fox&#8221; (i.e. eschewing grand theories for attention to lots of messy details), strong domain knowledge, general cognitive ability, and high scores on &#8220;need for cognition,&#8221; &#8220;actively open-minded thinking,&#8221; and &#8220;cognitive reflection&#8221; scales.\n<\/p><\/blockquote>\n<p>And here&#8217;s Tetlock&#8217;s portrait of a good forecaster:<\/p>\n<blockquote><p>\n  Philosophic outlook:<\/p>\n<ul>\n<li>Cautious: Nothing is certain<\/li>\n<li>Humble: Reality is infinitely complex<\/li>\n<li>Nondeterministic: Whatever happens is not meant to be and does not have to happen<\/li>\n<\/ul>\n<p>  Abilities and thinking styles:<\/p>\n<ul>\n<li>Actively open-minded: Beliefs are hypotheses to be tested, not treasures to be protected<\/li>\n<li>Intelligent and knowledgeable, with a &#8216;need for cognition&#8217;: intellectually curious; enjoy puzzles and mental challenges<\/li>\n<li>Reflective: Introspective and self-critical<\/li>\n<li>Numerate: Comfortable with numbers<\/li>\n<\/ul>\n<p>  Methods of forecasting:<\/p>\n<ul>\n<li>Pragmatic: Not wedded to any idea or agenda<\/li>\n<li>Analytical: Capable of stepping back from the tip-of-your-nose perspective and considering other views<\/li>\n<li>Dragonfly-eyed: Value diverse views and synthesize them into their own<\/li>\n<li>Probabilistic: Judge using many grades of maybe<\/li>\n<li>Thoughtful updaters: When facts change, they change their minds<\/li>\n<li>Good intuitive psychologists: Aware of the value of checking thinking for cognitive and emotional biases<\/li>\n<\/ul>\n<p>  Work ethic:<\/p>\n<ul>\n<li>Growth mindset: Believe it&#8217;s possible to get better<\/li>\n<li>Grit: Determined to keep at it however long it takes<\/li>\n<\/ul>\n<\/blockquote>\n<h3><span id=\"spend-time-with-people-who-have-good-judgement\" class=\"toc-anchor\"><\/span>Spend time with people who have good judgement<\/h3>\n<p>I haven&#8217;t seen any research about this, but I expect that \u2013 as with many skills and mindsets \u2013 the best way to improve is to spend time with other people who exemplify them.<\/p>\n<p>Spending time with people who have great judgement can help you improve almost automatically, by giving you behaviours to model, making it easy and fun to practice, giving you immediate feedback, and so on.<\/p>\n<p>Anecdotally, many people I know say that what they found most helpful in improving their judgement was debating difficult questions with other people who have good judgement.<\/p>\n<h3><span id=\"learn-about-whatever-youre-trying-to-make-judgements-about\" class=\"toc-anchor\"><\/span>Learn about whatever you&#8217;re trying to make judgements about<\/h3>\n<p>Although experts are often not the best forecasters, all else equal, more domain knowledge seems to help. Luke from Open Philanthropy:<\/p>\n<blockquote><p>\n  Several studies suggest that accuracy can be boosted by having (or acquiring) domain expertise. A commonly-held hypothesis, which I find intuitively plausible, is that calibration training is especially helpful for improving calibration, and that domain expertise is helpful for improving resolution.\n<\/p><\/blockquote>\n<h3><span id=\"learn-about-when-not-to-use-your-judgement\" class=\"toc-anchor\"><\/span>Learn about when <em>not<\/em> to use your judgement<\/h3>\n<p>One of the other lessons of Tetlock&#8217;s work is that combining many forecasts usually increases accuracy, and the average of many forecasts is better than most individuals. This suggests that another way to make better judgements is to seek out other people&#8217;s estimates, and to sometimes use them rather than your personal impressions.<\/p>\n<p>Exactly how much weight to put on the average estimate compared to your own views is a difficult question.<\/p>\n<ul>\n<li>For topics like your choice of career, where you have a lot of information that others don&#8217;t, it makes sense to rely more on your own assessments. <\/li>\n<li>For broader issues, we still need people to work to develop their personal impressions so that the average view can be well informed (if everyone defers to everyone else, no research will ever be done). <\/li>\n<\/ul>\n<p>It&#8217;s also important to remember it can often still be worth <em>acting<\/em> on a contrarian position, so long as the upsides are much bigger than the downsides.<\/p>\n<p>I hope to write more about how to balance judgement and independent thinking in another post (for now, I&#8217;d recommend reading <a href=\"https:\/\/forum.effectivealtruism.org\/posts\/WKPd79PESRGZHQ5GY\/in-defence-of-epistemic-modesty\"><em>In defence of epistemic modesty<\/em><\/a> by Greg Lewis and the comments).<\/p>\n<h3><span id=\"learn-more-about-decision-making-and-good-thinking\" class=\"toc-anchor\"><\/span>Learn more about decision making and good thinking<\/h3>\n<ul>\n<li>Read more about <a href=\"https:\/\/80000hours.org\/career-guide\/how-to-be-successful\/#12-use-research-into-decision-making-to-think-better\">how to think better<\/a>.<\/p>\n<\/li>\n<li>\n<p>Listen to our <a href=\"https:\/\/80000hours.org\/podcast\/episodes\/philip-tetlock-forecasting-research\/\">podcast with Philip Tetlock<\/a>, which starts with a summary of his research.<\/p>\n<\/li>\n<li>\n<p>We also particularly recommend this article on <a href=\"https:\/\/www.cold-takes.com\/minimal-trust-investigations\/\">minimal trust investigations<\/a> for developing your judgement and thinking about when to defer.<\/p>\n<\/li>\n<li>\n<p>See all <a href=\"https:\/\/80000hours.org\/topic\/foundations\/reasoning-well\/decision-making\/\/\">our articles and podcasts on decision-making<\/a>.<\/p>\n<\/li>\n<\/ul>\n<h2><span id=\"read-next\" class=\"toc-anchor\"><\/span>Read next<\/h2>\n<p>This is a supporting article in our <a href=\"\/advanced-series\/\">advanced series<\/a>. Here are some others you might find interesting:<\/p>\n<ul class=\"list-cards list-no-bullet row display-flex !tw--mb-0 margin-top\">\n<li class=\"col-sm-12 padding-bottom-small\">\n<div class=\"card card--horizontal row \">\n<div class=\"col-sm-4 col--card-image sm:!tw--pr-0\">          <a href=\"https:\/\/80000hours.org\/podcast\/episodes\/philip-tetlock-forecasting-research\/\" class=\"card__anchor no-visited-styling\">\n<div class=\"card__image bg-gray-light\">            <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/80000hours.org\/wp-content\/uploads\/2019\/06\/smallerfox-1-720x448.jpg\" alt=\"Decorative post preview\"                       class=\"tw--w-full \" style=\"\"                      width=\"720\" height=\"448\">            <\/div>\n<p>          <\/a>        <\/div>\n<div class=\"col-sm-8\">\n<div class=\"card__title\">\n<h3  class=\"\"><span id=\"accurately-predicting-the-future-is-central-to-absolutely-everything\" class=\"toc-anchor\"><\/span><a href=\"https:\/\/80000hours.org\/podcast\/episodes\/philip-tetlock-forecasting-research\/\" class=\"card__anchor no-visited-styling\">Accurately predicting the future is central to absolutely everything.<\/a><\/h3>\n<\/div>\n<div class=\"card__actions\">      <a href=\"https:\/\/80000hours.org\/podcast\/episodes\/philip-tetlock-forecasting-research\/\" class=\"card__action no-visited-styling\">Read more<\/a>    <\/div><\/div><\/div>\n<\/li>\n<li class=\"col-sm-12 padding-bottom-small\">\n<div class=\"card card--horizontal row \">\n<div class=\"col-sm-4 col--card-image sm:!tw--pr-0\">          <a href=\"https:\/\/80000hours.org\/problem-profiles\/improving-institutional-decision-making\/\" class=\"card__anchor no-visited-styling\">\n<div class=\"card__image bg-gray-light\">            <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/80000hours.org\/wp-content\/uploads\/2017\/09\/52009484451_c926044722_k-1-1-720x448.jpg\" alt=\"Decorative post preview\"                       class=\"tw--w-full \" style=\"\"                      width=\"720\" height=\"448\">            <\/div>\n<p>          <\/a>        <\/div>\n<div class=\"col-sm-8\">\n<div class=\"card__title\">\n<h3  class=\"\"><span id=\"improving-institutional-decision-making\" class=\"toc-anchor\"><\/span><a href=\"https:\/\/80000hours.org\/problem-profiles\/improving-institutional-decision-making\/\" class=\"card__anchor no-visited-styling\">Improving institutional decision making<\/a><\/h3>\n<\/div>\n<div class=\"card__actions\">      <a href=\"https:\/\/80000hours.org\/problem-profiles\/improving-institutional-decision-making\/\" class=\"card__action no-visited-styling\">Read more<\/a>    <\/div><\/div><\/div>\n<\/li>\n<li class=\"col-sm-12 padding-bottom-small\">\n<div class=\"card card--horizontal row \">\n<div class=\"col-sm-4 col--card-image sm:!tw--pr-0\">          <a href=\"https:\/\/80000hours.org\/articles\/expected-value\/\" class=\"card__anchor no-visited-styling\">\n<div class=\"card__image bg-gray-light\">            <img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/80000hours.org\/wp-content\/uploads\/2021\/09\/Shinning_in_clouds_-_Makra_Peak-720x448.jpg\" alt=\"Decorative post preview\"                       class=\"tw--w-full \" style=\"\"                      width=\"720\" height=\"448\">            <\/div>\n<p>          <\/a>        <\/div>\n<div class=\"col-sm-8\">\n<div class=\"card__title\">\n<h3  class=\"\"><span id=\"expected-value-how-can-we-know-what-makes-a-difference-when-uncertain\" class=\"toc-anchor\"><\/span><a href=\"https:\/\/80000hours.org\/articles\/expected-value\/\" class=\"card__anchor no-visited-styling\">Expected value: how can we know what makes a difference when uncertain?<\/a><\/h3>\n<\/div>\n<div class=\"card__actions\">      <a href=\"https:\/\/80000hours.org\/articles\/expected-value\/\" class=\"card__action no-visited-styling\">Read more<\/a>    <\/div><\/div><\/div>\n<\/li>\n<\/ul>\n<div class=\"well visible-if-not-newsletter-subscriber margin-bottom margin-top padding-top-small padding-bottom-small\">\n<h3 class=\"no-toc\">Plus, join our newsletter and we&#8217;ll mail you a free book<\/h3>\n<p>Join our newsletter and we&#8217;ll send you a free copy of <em>The Precipice<\/em> \u2014 a book by philosopher Toby Ord about how to tackle the greatest threats facing humanity. <a href=\"https:\/\/80000hours.org\/free-book\/#giveaway-terms\">T&#038;Cs here<\/a>.<\/p>\n<form data-80k-object-id=\"\" data-80k-form-action=\"newsletter__subscribe\" action=\"\/\" method=\"post\" class=\"form-newsletter-signup form-newsletter-signup-step-1 margin-bottom-smaller\">\n<div class=\"mc-field-group input-group compact-input-group \"> <input type=\"email\" value=\"\" name=\"email\" required class=\"form-control email\" placeholder=\"Email address\" id=\"input_email\"> <span class=\"submit input-group-btn input-group-btn-right\"> <input type=\"submit\" id=\"mc-embedded-subscribe\" value=\"GET THE BOOK\" class=\"btn btn-primary \" \/> <\/span> <\/div>\n<div> <input name=\"_eightyk_action\" value=\"mailchimp_add_subscriber\" type=\"hidden\"> <input name=\"redirect_path_after_step_2\" value=\"\/newsletter\/welcome\/\" type=\"hidden\"> <\/div>\n<div style=\"position: absolute; left: -5000px;\"> <input type=\"text\" name=\"b_abc12f58bbe8075560abdc5b7_43bc1ae55c\" tabindex=\"-1\" value=\"\"> <\/div>\n<\/form>\n<\/div>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":3,"featured_media":70798,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"[fn gather] Though we can still gather data about some risk factors -- e.g. number of wars, number of smaller disasters, etc.[\/fn]"},"categories":[323,1193,1217],"class_list":["post-70767","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ability","category-forecasting-research-implementation","category-reasoning-well"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Notes on good judgement and how to develop it - 80,000 Hours<\/title>\n<meta name=\"description\" content=\"Good judgement is the ability to weigh complex information and is a valuable career asset. Find out what we know about how to develop good judgement.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Notes on good judgement and how to develop it\" \/>\n<meta property=\"og:description\" content=\"The ability to weigh complex information is essential and can be developed.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/\" \/>\n<meta property=\"og:site_name\" content=\"80,000 Hours\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/80000Hours\" \/>\n<meta property=\"article:published_time\" content=\"2020-09-09T21:08:49+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-09-03T18:34:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/image5-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"796\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Benjamin Todd\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"Notes on good judgement and how to develop it\" \/>\n<meta name=\"twitter:description\" content=\"The ability to weigh complex information is essential and can be developed.\" \/>\n<meta name=\"twitter:creator\" content=\"@80000hours\" \/>\n<meta name=\"twitter:site\" content=\"@80000hours\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Benjamin Todd\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"16 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/\"},\"author\":{\"name\":\"Benjamin Todd\",\"@id\":\"https:\/\/80000hours.org\/#\/schema\/person\/a8603d1fe70a0a3de2c88b997d288bbe\"},\"headline\":\"Notes on good judgement and how to develop&nbsp;it\",\"datePublished\":\"2020-09-09T21:08:49+00:00\",\"dateModified\":\"2024-09-03T18:34:28+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/\"},\"wordCount\":3224,\"publisher\":{\"@id\":\"https:\/\/80000hours.org\/#organization\"},\"image\":{\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/fox1-2.png\",\"articleSection\":[\"Ability\",\"Forecasting research &amp; implementation\",\"Reasoning well\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/\",\"url\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/\",\"name\":\"Notes on good judgement and how to develop it - 80,000 Hours\",\"isPartOf\":{\"@id\":\"https:\/\/80000hours.org\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/fox1-2.png\",\"datePublished\":\"2020-09-09T21:08:49+00:00\",\"dateModified\":\"2024-09-03T18:34:28+00:00\",\"description\":\"Good judgement is the ability to weigh complex information and is a valuable career asset. Find out what we know about how to develop good judgement.\",\"breadcrumb\":{\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#primaryimage\",\"url\":\"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/fox1-2.png\",\"contentUrl\":\"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/fox1-2.png\",\"width\":4800,\"height\":2520},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/80000hours.org\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Advanced series\",\"item\":\"https:\/\/80000hours.org\/advanced-series\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Notes on good judgement and how to develop&nbsp;it\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/80000hours.org\/#website\",\"url\":\"https:\/\/80000hours.org\/\",\"name\":\"80,000 Hours\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/80000hours.org\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/80000hours.org\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/80000hours.org\/#organization\",\"name\":\"80,000 Hours\",\"url\":\"https:\/\/80000hours.org\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/80000hours.org\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/80000hours.org\/wp-content\/uploads\/2018\/07\/og-logo_0.png\",\"contentUrl\":\"https:\/\/80000hours.org\/wp-content\/uploads\/2018\/07\/og-logo_0.png\",\"width\":1500,\"height\":785,\"caption\":\"80,000 Hours\"},\"image\":{\"@id\":\"https:\/\/80000hours.org\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/80000Hours\",\"https:\/\/x.com\/80000hours\",\"https:\/\/www.youtube.com\/user\/eightythousandhours\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/80000hours.org\/#\/schema\/person\/a8603d1fe70a0a3de2c88b997d288bbe\",\"name\":\"Benjamin Todd\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/80000hours.org\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/564246602ba575c93f06c803b0b42df6?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/564246602ba575c93f06c803b0b42df6?s=96&d=mm&r=g\",\"caption\":\"Benjamin Todd\"},\"description\":\"Benjamin is the president and cofounder of 80,000 Hours. He managed the organisation while it grew from a lecture, to a student society, to the organisation it is today. He also helped to get effective altruism started in Oxford in 2011. This page lists all of his writing at 80,000 Hours. See [some highlights](https:\/\/benjamintodd.org\/#writing). To get a biweekly update that includes Benjamin's new articles, join our newsletter: [form_newsletter] To follow all his other writing and research ideas, [follow him on Twitter](https:\/\/twitter.com\/ben_j_todd). To learn more about him, see [Benjamin's personal website](https:\/\/benjamintodd.org\/), [Instagram](https:\/\/www.instagram.com\/benbentodd\/) or [LinkedIn bio](https:\/\/www.linkedin.com\/in\/benjamin-j-todd\/).\",\"sameAs\":[\"benjamin_todd_GcHF\"],\"url\":\"https:\/\/80000hours.org\/author\/benjamin-todd\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Notes on good judgement and how to develop it - 80,000 Hours","description":"Good judgement is the ability to weigh complex information and is a valuable career asset. Find out what we know about how to develop good judgement.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/","og_locale":"en_US","og_type":"article","og_title":"Notes on good judgement and how to develop it","og_description":"The ability to weigh complex information is essential and can be developed.","og_url":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/","og_site_name":"80,000 Hours","article_publisher":"https:\/\/www.facebook.com\/80000Hours","article_published_time":"2020-09-09T21:08:49+00:00","article_modified_time":"2024-09-03T18:34:28+00:00","og_image":[{"width":1200,"height":796,"url":"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/image5-1.png","type":"image\/png"}],"author":"Benjamin Todd","twitter_card":"summary_large_image","twitter_title":"Notes on good judgement and how to develop it","twitter_description":"The ability to weigh complex information is essential and can be developed.","twitter_creator":"@80000hours","twitter_site":"@80000hours","twitter_misc":{"Written by":"Benjamin Todd","Est. reading time":"16 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#article","isPartOf":{"@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/"},"author":{"name":"Benjamin Todd","@id":"https:\/\/80000hours.org\/#\/schema\/person\/a8603d1fe70a0a3de2c88b997d288bbe"},"headline":"Notes on good judgement and how to develop&nbsp;it","datePublished":"2020-09-09T21:08:49+00:00","dateModified":"2024-09-03T18:34:28+00:00","mainEntityOfPage":{"@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/"},"wordCount":3224,"publisher":{"@id":"https:\/\/80000hours.org\/#organization"},"image":{"@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#primaryimage"},"thumbnailUrl":"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/fox1-2.png","articleSection":["Ability","Forecasting research &amp; implementation","Reasoning well"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/","url":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/","name":"Notes on good judgement and how to develop it - 80,000 Hours","isPartOf":{"@id":"https:\/\/80000hours.org\/#website"},"primaryImageOfPage":{"@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#primaryimage"},"image":{"@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#primaryimage"},"thumbnailUrl":"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/fox1-2.png","datePublished":"2020-09-09T21:08:49+00:00","dateModified":"2024-09-03T18:34:28+00:00","description":"Good judgement is the ability to weigh complex information and is a valuable career asset. Find out what we know about how to develop good judgement.","breadcrumb":{"@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/80000hours.org\/2020\/09\/good-judgement\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#primaryimage","url":"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/fox1-2.png","contentUrl":"https:\/\/80000hours.org\/wp-content\/uploads\/2020\/09\/fox1-2.png","width":4800,"height":2520},{"@type":"BreadcrumbList","@id":"https:\/\/80000hours.org\/2020\/09\/good-judgement\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/80000hours.org\/"},{"@type":"ListItem","position":2,"name":"Advanced series","item":"https:\/\/80000hours.org\/advanced-series\/"},{"@type":"ListItem","position":3,"name":"Notes on good judgement and how to develop&nbsp;it"}]},{"@type":"WebSite","@id":"https:\/\/80000hours.org\/#website","url":"https:\/\/80000hours.org\/","name":"80,000 Hours","description":"","publisher":{"@id":"https:\/\/80000hours.org\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/80000hours.org\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/80000hours.org\/#organization","name":"80,000 Hours","url":"https:\/\/80000hours.org\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/80000hours.org\/#\/schema\/logo\/image\/","url":"https:\/\/80000hours.org\/wp-content\/uploads\/2018\/07\/og-logo_0.png","contentUrl":"https:\/\/80000hours.org\/wp-content\/uploads\/2018\/07\/og-logo_0.png","width":1500,"height":785,"caption":"80,000 Hours"},"image":{"@id":"https:\/\/80000hours.org\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/80000Hours","https:\/\/x.com\/80000hours","https:\/\/www.youtube.com\/user\/eightythousandhours"]},{"@type":"Person","@id":"https:\/\/80000hours.org\/#\/schema\/person\/a8603d1fe70a0a3de2c88b997d288bbe","name":"Benjamin Todd","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/80000hours.org\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/564246602ba575c93f06c803b0b42df6?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/564246602ba575c93f06c803b0b42df6?s=96&d=mm&r=g","caption":"Benjamin Todd"},"description":"Benjamin is the president and cofounder of 80,000 Hours. He managed the organisation while it grew from a lecture, to a student society, to the organisation it is today. He also helped to get effective altruism started in Oxford in 2011. This page lists all of his writing at 80,000 Hours. See [some highlights](https:\/\/benjamintodd.org\/#writing). To get a biweekly update that includes Benjamin's new articles, join our newsletter: [form_newsletter] To follow all his other writing and research ideas, [follow him on Twitter](https:\/\/twitter.com\/ben_j_todd). To learn more about him, see [Benjamin's personal website](https:\/\/benjamintodd.org\/), [Instagram](https:\/\/www.instagram.com\/benbentodd\/) or [LinkedIn bio](https:\/\/www.linkedin.com\/in\/benjamin-j-todd\/).","sameAs":["benjamin_todd_GcHF"],"url":"https:\/\/80000hours.org\/author\/benjamin-todd\/"}]}},"_links":{"self":[{"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/posts\/70767"}],"collection":[{"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/comments?post=70767"}],"version-history":[{"count":0,"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/posts\/70767\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/media\/70798"}],"wp:attachment":[{"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/media?parent=70767"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/80000hours.org\/wp-json\/wp\/v2\/categories?post=70767"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}