The wiki for the Open Notebook Science Challenge that I proposed during my UK trip is now available. We are currently looking for sponsors and participants.

Open Notebook Science Challenge

What?

The first round of this challenge calls upon groups or individuals with access to materials and equipment to measure the solubility of compounds in organic solvents and report their findings using Open Notebook Science .

Why?

Understanding exactly how an experiment was performed is essential to the efficient progress of science. There are no absolute facts in the scientific literature; every measurement reported is only meaningful within the full context of how it was generated. The purpose of a laboratory notebook is to report as much of this context as is reasonable. But to find trends data must be abstracted to a level where they can be manipulated in tables and charts. This is not a problem as long as one can drill down from each data point in a chart to the full context found in the laboratory notebook.

For example, a Google search for "vanillin solubility in THF" pulls up a lab book page EXP207 where it is reported to be 3.89M. This number might be used in a table of someone trying to quantify trends or test a mathematical model, in which case reliability of the number is important. By reading the lab notebook page it becomes clear that 118.5 mg of solid was measured on a scale with 0.1 mg accuracy. However only one measurement was obtained. All kinds of other details which might be important are provided, for example how long the mixture was vortexed, at what temperature and the physical appearance after evaporation. If this number turns out to be an outlier, one can investigate if a calculation error was the cause by inspecting the linked spreadsheet.

However, if a researcher is simply looking for the feasibility of making up a 2M solution of vanillin in THF for a reaction the margin of acceptable error is so wide that the answer is almost certainly "yes".

The purpose of Open Notebook Science is to allow immediate communication of scientific results. The value of these results will depend upon the quality of the laboratory notebook and the linked raw data. Publication in peer-reviewed journals is still an extremely important part of this process but it is not an appropriate vehicle for the efficient communication of this type of information.

In fact, one of the motivations for participating in this project is that we will collect data from sufficiently well recorded experiments and publish them in a peer-reviewed journal with the participation of the researchers as co-authors. We aim to build a mathematical model to predict solubility using the results obtained from this project.

Who?

Organizers

Jean-Claude Bradley
Cameron Neylon
Rajarshi Guha (modeling)

How?

Simply request an account on this wiki and start recording experiments using a format similar to UC-EXP207 . The organizers will provide feedback in the form of comments in bold and italics directly on the wiki. Hitting the Recent Changes link on the left navigation bar is a good way to keep track of edits.

When?

Now!

Modularizing Results and Analysis in Chemistry

Chemical research has traditionally been organized in either experiment-centric or molecule-centric models.

This makes sense from the chemist's standpoint.

When we think about doing chemistry, we conceptualize experiments as the fundamental unit of progress. This is reflected in the laboratory notebook, where each page is an experiment, with an objective, a procedure, the results, their analysis and a final conclusion optimally directly answering the stated objective.

When we think about searching for chemistry, we generally imagine molecules and transformations. This is reflected in the search engines that are available to chemists, with most allowing at least the drawing or representation of a single molecule or class of molecules (via substructure searching).

But these are not the only perspectives possible.

What would chemistry look like from a results-centric view?

Lets see with a specific example. Take EXP150, where we are trying to synthesize a Ugi product as a potential anti-malarial agent and identify Ugi products that crystallize from their reaction mixture.

If we extract the information contained here based on individual results, something very interesting happens. By using some standard representation for actions we can come up with something that looks like it should be machine readable without much difficulty:

• ADD container (type=one dram screwcap vial)
• ADD methanol (InChIKey=OKKJLVBELUTLKV-UHFFFAOYAX, volume=1 ml)
• WAIT (time=15 min)
• ADD benzylamine (InChIKey=WGQKYBSKWIADBV-UHFFFAOYAL, volume=54.6 ul)
• VORTEX (time=15 s)
• WAIT (time=4 min)
• ADD phenanthrene-9-carboxaldehyde (InChIKey=QECIGCMPORCORE-UHFFFAOYAE, mass=103.1 mg)
• VORTEX (time=4 min)
• WAIT (time=22 min)
• ADD crotonic acid (InChIKey=LDHQCZJRKDOVOX-JSWHHWTPCJ, mass=43.0 mg)
• VORTEX (time=30 s)
• WAIT (time=14 min)
• ADD tert-butyl isocyanide (InChIKey=FAGLEPBREOXSAC-UHFFFAOYAL, volume=56.5 ul)
• VORTEX (time=5.5 min)
• TAKE PICTURE

It turns out that for this CombiUgi project very few commands are required to describe all possible actions:

• ADD
• WAIT
• VORTEX
• CENTRIFUGE
• DECANT
• TAKE PICTURE
• TAKE NMR

By focusing on each result independently, it no longer matters if the objective of the experiment was reached or if the experiment was aborted at a later point.

Also, if we recorded chemistry this way we could do searches that are currently not possible:

• What happens (pictures, NMRs) when an amine and an aromatic aldehyde are mixed in an alcoholic solvent for more than 3 hours with at least 15 s vortexing after the addition of both reagents?
• What happens (picture, NMRs) when an isonitrile, amine, aldehyde and carboxylic acid are mixed in that specific order, with at least 2 vortexing steps of any duration?

I am not sure if we can get to that level of query control, but ChemSpider will investigate representing our results in a database in this way to see how far we can get.

Note that we can't represent everything using this approach. For example observations made in the experiment log don't show up here, as well as anything unexpected. Therefore, at least as long as we have human beings recording experiments, we're going to continue to use the wiki as the official lab notebook of my group. But hopefully I've shown how we can translate from freeform to structured format fairly easily.

Now one reason I think that this is a good time to generate results-centric databases is the inevitable rise of automation. It turns out that it is difficult for humans to record an experiment log accurately. (Take a look at the lab notebooks in a typical organic chemistry lab - can you really reproduce all those experiments without talking to the researcher?)

But machines are good at recording dates and times of actions and all the tedious details of executing a protocol. This is something that we would like to address in the automation component of our next proposal.

Does that mean that machines will replace chemists in the near future? Not any more than calculators have replaced mathematicians. I think that automating result production will leave more time for analysis, which is really the test of a true chemist (as opposed to a technician).

Here is an example of an analysis module making a simple point, useful to the chemistry community, and linking back to result modules that ultimately link back to the original experiment in the online laboratory notebook:

Context: obtaining precipitates in the CombiUgi project

Ugi reactions in methanol where the solution is supersaturated with Ugi product may give false negatives for precipitation. For example, a Ugi product rapidly crystallized at the 17th hour (RESULT0003) after addition of all reagents, while appearing as a clear solution at the 15th hour (RESULT0002). It is therefore recommended that the vials be submitted to vortexing (15 s) prior to taking a picture.

We'll be recording these analysis and result modules on UsefulChem wiki pages:

• http://usefulchem.wikispaces.com/ANALYSIS0001
• http://usefulchem.wikispaces.com/RESULT0001
• http://usefulchem.wikispaces.com/RESULT0002
• http://usefulchem.wikispaces.com/RESULT0003

We'll be using InChIKeys for compact unambiguous identification of molecules (and convenient indexing in Google) and the terms in this post for action options. Anyone is free to automatically incorporate these in a database, as long as attribution is provided. (If anyone knows of any accepted XML for experimental actions let me know and we'll adopt that.)

I think this takes us a step closer from freeform Open Notebook Science to the chemical semantic web, something that both Cameron Neylon and I have been discussing for a while now.

Chemistry Crowdsourcing with Open Notebook Science

I recently submitted a Letter of Intent for the NSF Cyber-Enabled Discovery and Innovation competition. Kevin Owens is a co-PI and will assist with the laboratory automation component. ChemSpider will contribute the database support. The pre-proposal is due in early January 2008 and we'll be writing it openly here. Comments are welcome.

We would ultimately like to enable the chemistry community to directly control the actions of a robot to help us understand some chemistry problems. As we make our way towards this goal, it would be very useful to start with suggestions for protocols to be executed by students we currently have in the group.

We already have a mechanism in UsefulChem to post experimental plans. In order to make the transition to full automation easier, it would be preferable if suggested protocols are even more specific than what we currently have listed. For example, instead of describing a general procedure like EXPLAN005, actually specify all of the compounds, amounts, mixing times, etc. This way the protocol can just be copied and pasted in the Procedure section of a new experiment, executed faithfully and reported in the main experiment list.

The main puzzle to solve is the prediction of which Ugi products will precipitate. A hypothesis might be that a precipitate will always occur from methanol at a certain minimal concentration of a certain reagent. Another approach might be based on the predicted molecular descriptors of the Ugi products. We might also start with as few assumptions as possible and use a genetic algorithm to evolve a solution. We'll be doing some of these but clearly there are more ways to solve this puzzle than we have resources or expertise.

So if anyone is interested in participating at this stage contact me to get access to the wiki and further discuss. Other examples of chemistry crowdsourcing : Chemmunity, The Synaptic Leap, OrgList, Chemists Without Borders and ChemUnPub.

Here is the LOI:

Chemistry Crowdsourcing using Open Notebook Science

The current system of dissemination of scientific data and knowledge is far less efficient than it needs to be to facilitate improved collaborative science, especially considering current publication vehicles and infrastructure. There is a growing movement promoting more Open Science, with the belief that a more transparent scientific process can perform far more effectively. The logical extension of this concept is full transparency - exposing a researcher's complete record of progress to the public in near real time. Not only will such a process enable ongoing data sharing it also provides an opportunity to develop collaborative communities of scientists and, at the conclusion of data acquisition, can enable communal extraction of conclusions when necessary. We have named this approach Open Notebook Science and have demonstrated its implementation and feasibility with the UsefulChem project, started in the summer of 2005, with the aim of synthesizing novel anti-malarial compounds. Our system currently uses free hosted services using general blog and wiki functions to facilitate replication across any scientific domains. These services are not chemically intelligent and are limited to text and graphic based data sharing only. For Open Notebook Chemistry the ability to intelligently manipulate, manage and search chemical structures and associated data is necessary and we have demonstrated proof of concept capabilities by integrating with the ChemSpider service, a free access online database managing chemical structures and focused on developing a structure centric community for chemists. This work will require the development of a chemically intelligent software platform to extend the capabilities of both the blog and the wiki environment for managing Open Notebook Science. The exposure of raw experimental procedures and data in a semantically rich format will enable the participation of both human and autonomous agents in the process of scientific discovery. This phenomenon of spontaneous group intelligence, referred to as "Crowdsourcing", has proven valuable in several contexts. Already, productive collaborations have been forged within the UsefulChem project with groups from Indiana University, Nanyang Technological University, the National Cancer Institute and UC San Francisco.

Surfing Web2O Article on the Chemistry Blogosphere

Richard Van Noorden of Chemistry World recently published a nice little review of Web 2.0 applications in chemistry. He introduced our UsefulChem project in a creative way:

As chemical reactions go, it was a complete failure. 'Contents of the reaction flask decomposed. Aborted', Drexel University chemist Jean-Claude Bradley and students recorded the day after an attempt to synthesise a catechol aldehyde from adrenaline on 24 January 2006. But the experiment has acquired a peculiar honour: Bradley chose it to be the first written in his group's new online laboratory notebook, in which all experimental data is made public and freely available to web users - a concept he later christened 'Open Notebook Science'.

Bradley's idea is simple: most failed experiments are discarded, yet their data could be useful to someone else. Even published papers don't always sufficiently explain the workings behind a successful experiment. In contrast, all Bradley's research and raw data is now documented transparently and almost in real-time. Anyone can see it, comment on it, and use it; and the internet is the perfect vehicle for hosting it.

There are many other examples in the article - a must read for the new generation of chemists.

Experimental Uncertainty Principle

Most of us are familiar with the mantra of how science progresses:

A hypothesis can never be completely proved by any finite set of experiments but it can be falsified by a single result.

In mathematical proofs, clear cut algorithms can usually be applied to prove unequivocally the falsehood of a theorem (notwithstanding Godel's incompleteness theorems :)

But in real research in the physical sciences, that is not exactly how scientists process reports of experimental results. And an important reason is the way results are reported.

Lets pick an example from the Open Access Beilstein Journal of Organic Chemistry.

Here is the full description of the experiment from the supplementary materials page:

To a solution of 5a (196 mg, 0.433 mmol) in CH2Cl2 (1.8 mL) was added p-toluenesulfonic acid (19 mg, 0.11 mmol). After stirring for 0.5 hours at 0 oC, the mixture was concentrated under reduced pressure and purified by flash chromatography on silica gel (eluent: ethyl acetate: P. E. = 1: 3) to provide 7 (152 mg, 100%) as a colorless oil.

I have omitted the characterization information. Lets assume for the moment that it is completely correct.

The question is : if 10 chemists follow this procedure as described, will they get 100% yield of pure product?

I think that it is quite possible that the results will vary wildly, including many complete failures. Here is why:

1) The reaction is carried out at 0 deg C for 30 minutes but the conditions of the work-up are completely unspecified. We don't know the pressure, the temperature of the bath or the duration of the solvent evaporation. The temperature of the rotovap bath will vary wildly from lab to lab, depending on vacuum pressure and personal preference. This is key because the conditions of the work-up (warmer and more concentrated) are much harsher than the reported reaction condition. My guess is that when this gets indexed in a database the reaction conditions will be further stripped of detail and likely end up as 0 C, 30 min.

2) The chromatography step does not specify how much silica to use, the dimensions of the column, the number of fractions, the TLC images of the fractions, the amount of solvent used to load the reaction mixture, etc. It may even be the case that the ratio of solvents was changed over the course of the chromatography - in a situation like this some would use a good solvent like methylene chloride to load the mixture then chase it with a solvent mixture containing a lower ethyl acetate/petroleum ether ratio.

3) A 100% isolated yield after chromatography means that not a single milligram was lost during transfer to the column and that all fractions containing the product were very pure. Ethyl acetate is notorious for increasing apparent product yields because it is sometimes difficult to remove on the vacuum pump. I would like to see the NMRs of the fractions.

This last point also brings up the issue of what the researcher does when confronted with an apparent 101% yield - since this is not chemically plausible it cannot be reported as such. Does the researcher state an assumption that there is a bit of extra solvent and slice off a milligram in the report? We can't tell from the information given in journals.

I want to make it clear that I am not picking on the authors for reporting in this way. Within the current norms of the organic chemistry community, this is an acceptable way to report laboratory procedures in peer reviewed journals.

Of course all (or most) of these details should have been recorded in the laboratory notebook. I understand that initially protocols in papers were abbreviated to save on space. But now with unlimited online supplementary materials associated with papers, researchers could scan their notebooks and all associated documents. But that is not required by the chemistry journals that I know and I have not seen it done.

Keep in mind that this is not new work - researchers already have (or should have) all of this as a routine part of doing research. This is one big advantage of Open Notebook Science - very little extra effort required. (Cameron Neylon also has a very nice recent summary of his thoughts on this.)

Any chemist will tell you (if they are honest) that there is almost always a mistake, however small in every experiment. By everyone agreeing to report experiments in a highly abbreviated form, it makes it convenient to get done more quickly and get that all important paper out the door. Do you completely start over an experiment because you measure 101% apparent yield? Or do you realize that you just don't have time and take a "shortcut" of some type to get that paper out.

All of this would go away if we came clean about our experiments - the good, bad and the ugly. Lets stop pretending that we did the reaction EXACTLY as stated in published abbreviated protocol and we might start to get out of this quagmire.

We don't have to change the way we abbreviate experiments - just link to the relevant pages in the laboratory notebook in the supplementary sections of papers.

As chemists try to make sense of the physical world and process results from other researchers, they have to evaluate the meaning of experiments published like this. Instead of processing the information algorithmically, they apply fuzzy logic: more weight is given to results with more proof.

With the limited information provided in this particular experimental description, I would expect that it is possible to get this reaction to work in good yield but I would not question the fundamental laws of nature if some chemists report that it fails completely. If I had access to the laboratory notebook and all raw data, including how the reaction was monitored, I would weigh the evidence of each report quite differently.

The more information one has about an experiment the more confidence one can place in the results. But it would never be possible to have complete confidence in any result, no matter how much information is provided. And because providing more information costs more in terms of time and money, a balance has to be struck.

We might call this the experimental uncertainty principle:

All experimental results are uncertain to some degree. Uncertainty can be reduced with more information but then fewer experiments can be performed with the same resources.

For example, an experiment like EXP064 provides extensive links to monitoring runs after each step in the reaction and provides evidence of the purity of the starting materials. By contrast EXP134 records 4 parallel reactions with only photographs as results. The purpose of the first experiment was to understand the Ugi reaction, while the second aims to quickly identify Ugi reagents that lead to easily purified products. When these reactions get compared, the second carries far less weight than the first - but we only know that by looking at the details in the notebook.

If we expect autonomous agents to contribute to the process of doing science (for example formulating and testing hypotheses), information has to be tagged in such a way that it incorporates a measure of uncertainty.

I suspect that it will be easier in many cases (like organic chemistry) to simply redo the experiment under known conditions rather than attempt to get hold of the original notebook.

Chemistry Corner on Drexel Island

With Eloise's help, we now have a Chemistry Corner on Drexel Island where chemists can put up posters and objects within Second Life.

We have different boards available at the store (multiple image Powerpoint-style, single image url link and single image teleport link) so it is really easy to set up a poster with little experience.
I have seeded the area with a presentation on Open Notebook Science and link to our UsefulChem wiki as well as a link to the Blue Obelisk wiki. The American Chemical Society will have a poster up shortly.

The Chemistry Corner is right next to the Ugi reaction staircase Jessica is building. (slurl)

CombiUgi: Time for Synthesis

For those who have not been following the progress of the CombiUgi project on our mailing list, here is a brief update.

The basic idea is to generate libraries of compounds that can be made quickly in the lab. The compounds in these virtual libraries are then prioritized according to potential usefulness (anti-malaria, anti-tumor, etc.). The top hits are then synthesized and tested.

We chose the Ugi reaction because it is very simple experimentally - mix four components (an amine, an aldehyde, a carboxylic acid and an isonitrile) in methanol at room temperature. In the creation of the libraries we used only components that were commercially available.

In our first library of 68,000 compounds, although all starting materials were commercially available, one of the main components needed (2-naphthyl isonitrile) for predicted anti-tumor activity was not available to ship for 6 weeks.

Since it is important to keep this science loop as short as possible we reran the library generation with Rajarshi Guha's open web service using only compounds that were next-day shippable. We also used this opportunity to increase the number of compounds in the virtual library to 500,000.

Rajarshi has now provided the docking results of this library with malarial enoyl reductase (PfENR). Here is what the top hit looks like:

So far we have been doing Ugi reactions usually with intensive NMR monitoring of each step. This has been very useful in debugging experimental conditions and reagent choice. Now that we have a better understanding of how to do it, generating many products quickly is more important than close monitoring.

We will take advantage of our observation that most of our Ugi products simply precipitate from solution over the course of a day. Instead of looking into additional purification techniques, we will make this an additional requirement for a compound to be considered a suitable lead. Any reactions that fail to precipitate within a day will be noted and discarded (see protocol). This information could surely be of use to others looking for suitable compound libraries.

As for the testing phase of these potential anti-malarial agents, I have a few very promising leads now and I'll report shortly when I get confirmation of a collaborator.

Indexing Molecules in Second Life

As I've recently commented, there has been media interest in the use of the virtual online world Second Life for chemistry. We also recently demonstrated on Drexel Island that it was possible to visualize molecular docking using the molecular rezzer developed by Andrew Lang.

Nature Island also hosts several common molecules, including buckyballs. As more people start to experiment with representing chemicals and chemistry research in Second Life it would be nice if such examples were discovered by a simple Google search.

All that really needs to be done to accomplish this is to co-locate molecular descriptors with corresponding SLURLs (Second Life URLs) on the same web page. When clicked, the SLURL will automatically start Second Life and teleport the user to the location where the molecule can be found. If the user does not have Second Life, a page pops up explaining how to set up a free account and download the software. This could be a good way to introduce the mainstream chemical community to new modalities of communicating science.

As for descriptors, I am suggesting that we use InChIs and common names at the very least. Google does a fairly good job of finding molecules by InChI.

I created a wiki,
http://secondlifemolecules.wikispaces.com/
and seeded it with a molecule from our malaria research that I've used in several places and with caffeine, which is displayed on Nature Island. I invite anyone to contribute to the wiki and add information that could be useful. (The indexing on Google can take a few days for a new wiki)

There are several other ways of creating this index and I think the more redundancy the better. For example, we could make Second Life a "supplier" on ChemSpider. It might also be possible for Andrew's molecule rezzer to note the location of a molecule when it gets created in Second Life and automatically send off an email to a Blogger account to create a post.



Chemical Blogspace Tags

InChI=1/C8H10N4O2/c1-10-4-9-6-5(10)7(13)12(3)8(14)11(6)2/h4H,1-3H3
caffeine

UsefulChem on Drexel Island

Today, I added a little section for UsefulChem on the northeast section of the Drexel Island in Second Life. There are slides from my presentation at the ACS on Open Notebook Science as well as a pic of one of our Ugi products docking in enoyl reductase. The 3D structure of the molecule is also floating there, ready to be rotated and inspected.

We're now working on getting the docking visualized in full 3D space. Eloise is helping with that now.

Don't be shy - come visit! (slurl)

Open Source Science Expands with Tan

A few months ago I posted a request for a docking collaborator to help us understand and reproduce our docking results with our targeted anti-malarial compounds. I've been contacted by a few people and I'll report on every one of those collaborations when it is possible.

Last week, Tan Tsu Soo from the Bioinformatics Group at Nanyang Polytechnic in Singapore informed me that he worked things out with his Director and was now in a position to collaborate with us openly. He has been busy docking uncyclized Ugi products and diketopiperazines using FlexX to compare with the results than Sean has obtained in D-EXP005.

He has not obtained identical results, partly because his search is not restricted to the same binding site that Sean used with THINK.

But the good news is that Tan also has managed to dock a good number of our molecules, including Ugi products that we have made and can start testing. He uploaded some of his results using Jmol to demonstrate the docking. Stay tuned for more data on the wiki.

The deprotected Ugi products (R and S) that Alicia prepared in EXP062 are shown below to dock in a similar fairly shallow location. (DRuncycltBu and DSuncycltBu from D-EXP005) Others, like the catechol derivative BRuncycltBu bind deep into the pocket. Check them out on Jmol.

Marketplace Segment on Open Notebook Science

The NPR interview on Open Science I discussed two weeks ago has aired and is now available.

I think it was very well balanced. The positive aspects of not losing failed experiments was weighed against the difficulties in publishing in some journals and of deriving profit.

Second Life at the ACS and Quizzes

Yesterday, I gave my first talk at the March07 ACS meeting on Teaching Organic Chemistry with Blogs and Wikis. The screencast is now available.

It was part of a symposium on Using Social Networking Tools to Teach Chemistry organized by Harry Pence and Andrea Gay. Joanna Scott gave a most interesting talk about Nature's experimentation with Second Life and the great possibilities for communicating research work. Harry is also involved with Second Life. Indeed I met him in world by accident a few days ago!

Largely because of Beth Ritter-Guth's tireless dedication to implementing educational opportunities in Second Life, I am finally coming around and seeing the potential for teaching and research. With the help of Eloise Pasteur (SL name), Beth has created an adaptation of the EduFrag quizzes for second life.

The rules are the same but the interface is very different from Unreal Tournament. Click on the obelisk to get the quiz started. Four images will appear and only one will be correct. Click on the correct one to go to the next set. Clicking on an incorrect image will make you start over. If you make it past the 20th room you will be a rewarded with a picture of my cat yawning.

The material in the current quiz is on introductory organic chemistry (Lewis structures, Newman projections, nomenclature, etc.) and I will make good use of it in the class I am teaching next week.

Give it a try and let me know how it works. The quiz is located in the Open Notebook Science building on Eduisland (slurl). We'll be adding more material related to UsefulChem there shortly (thus the use of the Blue Obelisk).

Code for Open Content

After my post on searching Google for documents that are free to share, Egon asked me about how to indicate that on our sites.

Wikispaces does it automatically and looking at the HTML it looks like you just need to add this tag (or whatever Creative Commons License you choose):

< a rel="license" href="http://www.creativecommons.org/licenses/by-sa/2.5">Creative Commons Attribution Share-Alike 2.5 License >

I've now added this to the template file of most of my blogs on Blogger.