Blog

The Future of Grant-making Funded by Jaan Tallinn at BERI

by on

BERI is planning to hand off our involvement in grant-making to 501(c)(3) organizations from the donations of philanthropist Jaan Tallinn. The hand-off will be to one or more other teams and/or processes that are separate from BERI. Andrew Critch, who has been instrumental in making grants funded by Jaan’s donations, will oversee this hand-off process, and will likely continue to act as an independent advisor to Jaan’s grant-making in the future. It is BERI’s understanding that Jaan will continue to sponsor grant-making in the area of existential risk reduction through other grant-making entities going forward.

Read more


Activity Update - November 2018-March 2019

by on

This is an update covering since our last blog post, November 2018 - March 2019. We appreciate your review, feedback, and/or advice.

Read more


Activity Update - October 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities (or on other activities) that may not be covered in these posts until our plans are more solid.

Read more


Announcing the Grantees from Round One of BERI's Project Grants Program

by on

This last June, BERI announced its first Individual Grants program: a Project Grants round. This post summarizes our process and announces the grant winners for this round.

Process

We received 55 applications by the end of June, when the round closed. We then spent a total of approximately 77 person hours reviewing the applications, contacting references, and consulting advisors.

In late August, the Project Grants Committee (comprising three anonymous individuals) finalized its recommendations and sent those to BERI’s First Grant Fund Committee (more on the FGF Committee at this link) and Board of Directors for approval. We then emailed applicants notifying them of their status. We have spent most of September and early October setting up the logistics for the grants (drafting and sending out grant agreements, and disbursing funding). We have not yet finalized the disbursement of all grants.

Common considerations

We thought it might be helpful for future applicants to lay out some of the considerations that the project grants committee frequently found itself discussing. The following factors are presented in no particular order, and each committee member put different weight on each factor:

  1. The applicant’s general and specific competencies as evidenced by their past achievements (e.g., educational achievements, business achievements, social achievements, etc.).
  1. The quality and similarity of the applicant’s past work samples to the project being proposed.
  1. The general and specific competencies of the applicant’s references, as well as the content of the recommendations that the references submitted.
  1. The applicant’s track record of commitment to long-term futurist causes, or other signs of likelihood that the applicant cared about the long-term future.
  1. The likely and potential impact of the project, as well as the particular cost-effectiveness of the project being proposed.
  1. Potential downsides of the project.
  1. Whether the grantee was a good fit for the parameters of this project grant round.
  1. What would happen if BERI did not fund the project right now.

However, note that the committee may put more weight on a different set of factors in the future.

Mistakes & Challenges

There were several kinks in our process (we expected this, given that it was our first individual grants round). Some examples:

  • We sent out unclear recommendation forms to references, resulting in more back-and-forth about the forms than we had anticipated (and likely leading to fewer references being submitted than otherwise would have).

  • The project grants committee had difficulties scheduling meetings given summer vacation schedules and other events.

  • There were a number of internal steps that needed further development during the evaluation process, such as:

    • Transforming information gathered by the application form into an easy-to-compare format for committee members,

    • Collecting additional information on some applications for committee members, and

    • Developing grant agreements and figuring out how to communicate to grantees about the tracking requirements for the grants.

These steps took more time than we anticipated, and developing them as we went (as opposed to ahead of time) likely led to somewhat sub-optimal solutions in many cases.

However, despite the challenges, we’re relatively satisfied with how the grants round went—we managed to mostly keep to the schedule we set for ourselves and are excited about our selections. We now have a long list of improvements that we’d like to make for future individual grants rounds.

Grant Winners

See the bottom of this page for a list of winners

Of the 55 applications, we selected 18 to receive funding (~33%). One applicant was awarded a vendor contract in lieu of a grant based on the project’s legal structure and the availability of services offered that BERI wanted to purchase (not included in the grants totals BERI is reporting). We managed to stay close to the budget we initially set for this first round ($750k); in total, we plan to award $799,465 to selected winners (excluding the vendor). We are not sure how many applicants or grant winners to expect for future rounds—we predict (with low confidence) that we will receive more applicants, maintain our budget, and therefore accept a smaller fraction of applicants in the future.

Our grant winners are listed on our grants page. Currently, only those who have received their grants are listed; we will continue to update the grants page as we finalize disbursing the grants.

We’re excited about our grantees, and we hope you are too! We believe that having diverse projects all pursing work on x-risk as an objective is important to helping develop a culture of focus on humanity’s long-term survival and flourishing.

List of winners:

Grantee Amount Purpose
Baeo Maltinsky $55,000 Baeo will further his research on AI and technology trends.
Ben Goldhaber $150,000 The grant will support his project (co-lead by Jacob Lagerros) to bring x-risk-relevant questions to popular prediction platforms.
Colleen McKenzie $51,000 The grant will support Colleen's research on AI timelines and the processes that produce technical and scientific progress.
David Manheim $25,000 David will research aspects of Goodhart’s law, focusing on multi-agent dynamics.
Jamison Bryce Hidysmith $20,500 Bryce will analyze global risks from technology through a geopolitical lens.
Jessica Taylor $10,000 Jessica will work on her research in AI alignment and other areas.
Jordan Alexander $1,900 Jordan will host several meetings at Stanford EA and the Stanford Transhumanist Association.
Justin Shovelain $100,000 Justin will lead and develop “Convergence Analysis”, a new group focused on x-risk strategy research.
Luca Rade $20,400 Luca will research the implications of coarse-graining by an agent in a complex environment for AI alignment.
Lucius Caviola $44,353 Lucius will conduct research on the psychology of existential risk.
Markus Stoor (via EA Sweden) $4,000 Effective Altruism Sweden will support Markus Stoor’s project to coordinate two follow-up lunch-to-lunch meetings in Sweden for x-risk-focused individuals.
Roxanne Heston $49,532 Roxanne will work on a variety of AI policy projects in Washington, D.C.
Sarah D Spikes $24,000 Sarah will implement improvements in support of the Rationality and Effective Altruism Community Hub (REACH).
Sebastian Farquhar $12,000 Sebastian will attend conferences and purchase compute for experiments related to his PhD research on uncertainty modeling in neural networks.
Stefan Schubert $86,580 Stefan will conduct research on the psychology of existential risk.
Stephanie Zolayvar $100,000 Stephanie will train herself in circling and host circles for people who are promising contributors to reducing x-risk.
Zoe Cremer $25,200 The grant will support Zoe as a visiting fellow at CFI, where she will research disagreements about the amount and kind of structure required for AGI

Questions or comments?

If you have feedback or concerns, please reach out to individual-grants@existence.org.


Activity Update - September 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - July & August 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


X-risk Relevant Prediction Market Question Suggestions (Medium Term)

by on

Background & motivation
The questions
    General performance benchmarks
    Will reinforcement learning be the dominant paradigm in n years?
    How well will reinforcement learning or its successor work?
    Operationalizations of AI relevant compute
    Architecture, modularity, and deep learning
        How many modules?
        Gradient descent modules vs. non-gradient descent modules?
        How many meta-modules?
    Sociological questions
    Governments and nationality
    Consensus in the ML community that the “alignment problem” is important

Read more


Activity Update - June

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - May

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Project Grants Program -
Round 1

by on

Update November 2018: Round 1 Project Grants have been awarded. See the announcement here.

Read more


Activity Update - April

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - February and March 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - January 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - December 2017

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Announcing BERI Computing Grants

by on

Update: As of June 5, 2018, this program is paused and no longer accepting applications. It has been superceded by BERI’s Project Grants program.

Read more


Activity Update - November 2017

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - October 2017

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Forming an engineering team

by on

BERI wants to help CS students and researchers at UC Berkeley who care about existential risk, by hiring a team of engineers to collaborate with them on their work. It might not be obvious why outside help is needed for this, and the purpose of this post is explain why.

Read more


What we’re thinking about as we grow - ethics, oversight, and getting things done

by on

BERI has only been around for nine months, but we’ve already learned about significantly more needs than our current capacity can support. We’ve encountered many ideas that we believe would be great to execute, such as providing additional computational resources to x-risk oriented PhD students, hiring writers and editors for influential publications, investigating grant opportunities for our grants program. We feel almost as if BERI’s existence has “raised an antenna” which receives requests for help on x-risk reduction projects—and those requests are coming in rapidly.

Read more


Activity Update - September 2017

by on

BERI has been shifting gears somewhat, and we thought it would be helpful to our collaborators and supporters to start a monthly blog post series. Each blog post will briefly describe some of what we’ve been up to each month and hopefully shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Announcing BERI’s first grants program

by on

Update: See our grants page for the latest information about our grants program.

Read more


BERI's semi-annual report, August 2017

by on | Reports

It’s been slightly more than six months since BERI was incorporated. In our first half year, we have:

  • Assisted the Center for Human-Compatible AI (CHAI) with several projects, including the coordination of CHAI’s first conference.
  • Hired two employees and eight contractors.
  • Raised over $400,000 for future projects, including activities that support CHAI.

Read more