Blog

BERI 2023 Annual Report

by on

We’ve released our 2023 annual report here.

Read more


2024 Collaborator Survey Results

by on

In January 2024 we sent a survey to 70 of our collaborators, attempting to measure their opinions of BERI. The survey was based off the previous collaborator survey sent in 2022. No questions were omitted from this year’s verision, but several new questions were added.

Read more


BERI Returns All FTX Funds

by on

We are happy to announce that as of January 31, 2024 BERI has returned funds received from the FTX Foundation (also referred to as the FTX Future Fund) in the amount of $389,745.

Read more


BERI’s Goals and Predictions for 2024

by on

Previous BERI annual reports have concluded with a list of predictions (2021, 2020) about BERI’s activities for the following year. Annual predictions are best made early in the year. In contrast, our 2022 annual report wasn’t released until July of 2023, the year we would have been predicting. To avoid that issue this year, we’ve opted to publish our 2024 predictions as a blog post instead.

Read more


OCPL and SRL Converted to Main Collaborations

by on

BERI has received grants in support of two of our trial collaborations:

Read more


New University Collaborators

by on

This year marked the fourth year that BERI has publicly solicited applications for new university collaborations. We had an unexpected doubling in the number of applications we received compared to any previous year, accompanied by an increase in the overall quality of those applications. As usual, applications came from a wide variety of academic disciplines and geographical areas, with the majority being outside of the US.

Read more


Applications open for new university collaborators

by on

The Berkeley Existential Risk Initiative (BERI) is once again accepting applications from university-affiliated research groups and individual academics interested in receiving our support. Applications are due August 31st.

Read more


BERI 2022 Annual Report

by on

We’ve released our 2022 annual report here.

Read more


CLTC converted to main collaboration

by on

BERI was recently recommended a $522,000 grant via the Survival and Flourishing Fund to support our collaboration with the Center for Long-Term Cybersecurity (CLTC) at UC Berkeley. This grant will support the ongoing work by Dr. Anthony Barrett and the team at CLTC on developing AI standards, in particular creating an AI risk management standards “profile” with supplemental guidance for developers of increasingly general-purpose AI.

Read more


New BERI Deputy Director: Elizabeth Cooper

by on

I’m excited to welcome Elizabeth Cooper as BERI’s new Deputy Director!

Read more


New University Collaborators

by on

We’ve completed our evaluation of the applications for new collaborators. We received a total of 19 applications from groups and individuals, across a wide variety of academic disciplines, geographical areas, and project types. After careful consideration, we’ve decided to start collaborations with the following groups:

Read more


Applications open for new university collaborators

by on

BERI is once again accepting applications from university-affiliated groups and individuals interested in receiving our support.

Read more


Funding for our collaboration with David Krueger's lab

by on

BERI has received a $40,000 grant from Open Philanthropy to support our collaboration with Professor David Krueger’s lab at the University of Cambridge. We originally launched a trial collaboration with David in July 2021, and with this donation we’re converting our trial collaboration with him into one of our main collaborations.

Read more


BERI 2021 Annual Report

by on

We’ve released our 2021 annual report here.

Read more


BERI 2022 collaborator survey results

by on

A summary of the results of our first formal collaborator survey is now available here. One of BERI’s main goals is to support university x-risk researchers in whatever way they need, so it’s useful for us to check whether those researchers actually think BERI is helping them. Surveys like this helps BERI to monitor and improve our impact.

Read more


What does BERI do and why is it important?

by on

I recently gave a presentation about BERI to the Global Priorities Institute at Oxford University. My preparations for the talk helped crystallize a few thoughts about BERI that I’ve had in rough form for a while. So I thought I’d take the more novel parts of the presentation and turn them into a blog post.

Read more


InterACT converted to main collaboration

by on

BERI has received a $135,000 grant from the Long-Term Future Fund to support our collaboration with the Interactive Autonomy and Collaborative Technologies Laboratory (InterACT) at UC Berkeley. We originally launched a trial collaboration with InterACT in August 2020, and with this donation we’re converting our trial collaboration with InterACT into one of our main collaborators.

Read more


Listing Our Donors Publicly

by on

Starting today, most of BERI’s donors will be listed on our transparency page.

Read more


Sculpting Evolution and ALL Converted to Main Collaborations

by on

BERI has received grants in support of two of our trial collaborations:

  1. A $250,000 grant from the Long-Term Future Fund to support our collaboration with the Autonomous Learning Laboratory (ALL) at the University of Massachusetts Amherst. We originally launched a trial collaboration with ALL in August 2020.
  2. A $30,000 grant from Open Philanthropy (via a recommendation to their DAF at SVCF) to support our collaboration with the Sculpting Evolution group at MIT. We originally launched a trial collaboration with Sculpting Evolution in July 2021.

Read more


New University Collaborators

by on

We’ve completed our evaluation of the applications for new collaborators. We received a total of 16 applications from groups and individuals, across a wide variety of academic disciplines, geographical areas, and project types. After careful consideration, we’ve decided to start collaborations with the following groups:

Read more


Board changes

by on

Eric Rogstad has stepped down from BERI’s Board of Directors. Eric has been with BERI since the very beginning, and BERI would not exist in its current form without his contributions. I have personally benefitted greatly from his insights and advice.

Read more


New University Collaborators

by on

BERI is once again accepting applications from university-affiliated groups and individuals interested in receiving our support.

Read more


Trial collaborations

by on

The purpose of this blog post is to explain BERI’s “trial collaborations”: why they exist, how they differ from “main collaborations,” and the role they play in BERI’s plans for expansion.

Read more


SERI converted to main collaboration

by on

BERI has received a $60,000 grant from the Long-Term Future Fund to support our collaboration with the Stanford Existential Risks Initiative (SERI). We launched a trial collaboration with SERI in August 2020, and with this donation we’re converting our collaboration with SERI from a trial into one of our main collaborators.

Read more


BERI 2020 Annual Report

by on

We’ve released our 2020 annual report here.

Read more


BERI Overhead

by on

This post explains how BERI thinks about per-project overhead rates, and how we raise funds to support our core operations.

Read more


Fundraiser success

by on

Thanks to the generosity of our donors, we raised $60,702 as part of our December fundraiser, exceeding our goal of $50,000! These funds will go towards launching new trial collaborations in 2021, as described here.

Read more


BERI 2020 Fundraiser

by on

Update 2020-01-08: Fundraiser success!

Read more


New University Collaborators

by on

We’ve completed our evaluation of the applications for new collaborators. We received a total of 19 applications from groups and individuals, across a wide variety of academic disciplines, geographical areas, and project types. After careful consideration, we’ve decided to start collaborations with the following groups:

Read more


New Collaborator Applications

by on

BERI is expanding our offerings to provide free services to a wider set of university-affiliated groups and projects, and we’re now accepting applications from groups and individuals interested in receiving our support. Particularly in light of COVID-19, we’re standing ready to help people try out new ways of getting things done.

Read more


Board and Staff Changes

by on

At the most recent meeting of BERI’s Board of Directors, Sawyer Bernath was promoted to Deputy Director of BERI, and joined the Board. I’m excited about Sawyer’s increasingly central role at BERI, which is key to our “distillation” plan for 2020, as described in this blog post.

Read more


BERI's Plans for 2020

by on

In 2020, BERI’s main focus will be almost exclusively on its collaborations program, which supports projects with CHAI, CSER, and FHI. I think collaboration with universities remains BERI’s greatest potential value-add to the world, and it’s the capacity that I’m most excited to see BERI maintain and expand. We will try to keep up with the existing responsibilities of BERI’s other programs, but will not be taking on new responsibilities, and are generally aiming to wind down non-collaboration activities. See this post for more information about the winding down of BERI’s grants program.

Read more


The Future of Grant-making Funded by Jaan Tallinn at BERI

by on

BERI is planning to hand off our involvement in grant-making to 501(c)(3) organizations from the donations of philanthropist Jaan Tallinn. The hand-off will be to one or more other teams and/or processes that are separate from BERI. Andrew Critch, who has been instrumental in making grants funded by Jaan’s donations, will oversee this hand-off process, and will likely continue to act as an independent advisor to Jaan’s grant-making in the future. It is BERI’s understanding that Jaan will continue to sponsor grant-making in the area of existential risk reduction through other grant-making entities going forward.

Read more


Activity Update - November 2018-March 2019

by on

This is an update covering since our last blog post, November 2018 - March 2019. We appreciate your review, feedback, and/or advice.

Read more


Activity Update - October 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities (or on other activities) that may not be covered in these posts until our plans are more solid.

Read more


Announcing the Grantees from Round One of BERI's Project Grants Program

by on

This last June, BERI announced its first Individual Grants program: a Project Grants round. This post summarizes our process and announces the grant winners for this round.

Process

We received 55 applications by the end of June, when the round closed. We then spent a total of approximately 77 person hours reviewing the applications, contacting references, and consulting advisors.

In late August, the Project Grants Committee (comprising three anonymous individuals) finalized its recommendations and sent those to BERI’s First Grant Fund Committee (more on the FGF Committee at this link) and Board of Directors for approval. We then emailed applicants notifying them of their status. We have spent most of September and early October setting up the logistics for the grants (drafting and sending out grant agreements, and disbursing funding). We have not yet finalized the disbursement of all grants.

Common considerations

We thought it might be helpful for future applicants to lay out some of the considerations that the project grants committee frequently found itself discussing. The following factors are presented in no particular order, and each committee member put different weight on each factor:

  1. The applicant’s general and specific competencies as evidenced by their past achievements (e.g., educational achievements, business achievements, social achievements, etc.).
  1. The quality and similarity of the applicant’s past work samples to the project being proposed.
  1. The general and specific competencies of the applicant’s references, as well as the content of the recommendations that the references submitted.
  1. The applicant’s track record of commitment to long-term futurist causes, or other signs of likelihood that the applicant cared about the long-term future.
  1. The likely and potential impact of the project, as well as the particular cost-effectiveness of the project being proposed.
  1. Potential downsides of the project.
  1. Whether the grantee was a good fit for the parameters of this project grant round.
  1. What would happen if BERI did not fund the project right now.

However, note that the committee may put more weight on a different set of factors in the future.

Mistakes & Challenges

There were several kinks in our process (we expected this, given that it was our first individual grants round). Some examples:

  • We sent out unclear recommendation forms to references, resulting in more back-and-forth about the forms than we had anticipated (and likely leading to fewer references being submitted than otherwise would have).

  • The project grants committee had difficulties scheduling meetings given summer vacation schedules and other events.

  • There were a number of internal steps that needed further development during the evaluation process, such as:

    • Transforming information gathered by the application form into an easy-to-compare format for committee members,

    • Collecting additional information on some applications for committee members, and

    • Developing grant agreements and figuring out how to communicate to grantees about the tracking requirements for the grants.

These steps took more time than we anticipated, and developing them as we went (as opposed to ahead of time) likely led to somewhat sub-optimal solutions in many cases.

However, despite the challenges, we’re relatively satisfied with how the grants round went—we managed to mostly keep to the schedule we set for ourselves and are excited about our selections. We now have a long list of improvements that we’d like to make for future individual grants rounds.

Grant Winners

See the bottom of this page for a list of winners

Of the 55 applications, we selected 18 to receive funding (~33%). One applicant was awarded a vendor contract in lieu of a grant based on the project’s legal structure and the availability of services offered that BERI wanted to purchase (not included in the grants totals BERI is reporting). We managed to stay close to the budget we initially set for this first round ($750k); in total, we plan to award $799,465 to selected winners (excluding the vendor). We are not sure how many applicants or grant winners to expect for future rounds—we predict (with low confidence) that we will receive more applicants, maintain our budget, and therefore accept a smaller fraction of applicants in the future.

Our grant winners are listed on our grants page. Currently, only those who have received their grants are listed; we will continue to update the grants page as we finalize disbursing the grants.

We’re excited about our grantees, and we hope you are too! We believe that having diverse projects all pursing work on x-risk as an objective is important to helping develop a culture of focus on humanity’s long-term survival and flourishing.

List of winners:

Grantee Amount Purpose

Questions or comments?

If you have feedback or concerns, please reach out to individual-grants@existence.org.


Activity Update - September 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - July & August 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


X-risk Relevant Prediction Market Question Suggestions (Medium Term)

by on

Background & motivation
The questions
    General performance benchmarks
    Will reinforcement learning be the dominant paradigm in n years?
    How well will reinforcement learning or its successor work?
    Operationalizations of AI relevant compute
    Architecture, modularity, and deep learning
        How many modules?
        Gradient descent modules vs. non-gradient descent modules?
        How many meta-modules?
    Sociological questions
    Governments and nationality
    Consensus in the ML community that the “alignment problem” is important

Read more


Activity Update - June

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - May

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Project Grants Program -
Round 1

by on

Update November 2018: Round 1 Project Grants have been awarded. See the announcement here.

Read more


Activity Update - April

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - February and March 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - January 2018

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - December 2017

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Announcing BERI Computing Grants

by on

Update: As of June 5, 2018, this program is paused and no longer accepting applications. It has been superceded by BERI’s Project Grants program.

Read more


Activity Update - November 2017

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Activity Update - October 2017

by on

This post is part of our monthly activity update series. Each activity update briefly describes what we’ve been up to in the last month and, hopefully, helps to shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Forming an engineering team

by on

BERI wants to help CS students and researchers at UC Berkeley who care about existential risk, by hiring a team of engineers to collaborate with them on their work. It might not be obvious why outside help is needed for this, and the purpose of this post is explain why.

Read more


What we’re thinking about as we grow - ethics, oversight, and getting things done

by on

BERI has only been around for nine months, but we’ve already learned about significantly more needs than our current capacity can support. We’ve encountered many ideas that we believe would be great to execute, such as providing additional computational resources to x-risk oriented PhD students, hiring writers and editors for influential publications, investigating grant opportunities for our grants program. We feel almost as if BERI’s existence has “raised an antenna” which receives requests for help on x-risk reduction projects—and those requests are coming in rapidly.

Read more


Activity Update - September 2017

by on

BERI has been shifting gears somewhat, and we thought it would be helpful to our collaborators and supporters to start a monthly blog post series. Each blog post will briefly describe some of what we’ve been up to each month and hopefully shed light on how our priorities shift over time. Note that some of our time each month will be spent investigating future project opportunities that may not be covered in these posts until our plans are more solid.

Read more


Announcing BERI’s first grants program

by on

Update: See our grants page for the latest information about our grants program.

Read more


BERI's semi-annual report, August 2017

by on | Reports

It’s been slightly more than six months since BERI was incorporated. In our first half year, we have:

  • Assisted the Center for Human-Compatible AI (CHAI) with several projects, including the coordination of CHAI’s first conference.
  • Hired two employees and eight contractors.
  • Raised over $400,000 for future projects, including activities that support CHAI.

Read more