Grade:💩 This paper published yesterday analyzes T cell immunity against COVID-19: https://t.co/d9VJ3avFcW We disc… https://t.co/ElKmFHCyOe
The authors analyzed blood samples from 206 individuals among 7 groups: 17 acute cases, severe (AS) 10 acute cases… https://t.co/XiN58MpBT7
55 healthy unexposed blood donors from May 2020, during pandemic (2020_BD) 28 healthy unexposed blood donors from… https://t.co/Ofh9idjonp
Figure 3A claims about 20%/55%/70%/85% of 2020_BD/Exp/MC/SC cases elicit T cell responses, which directly conflicts… https://t.co/iUVP0ybZvu
Figure 4G claims 61% of Exp cases are seropositive, directly conflicting with page 27 claiming 64%. Small differenc… https://t.co/040mswfHCE
The mini figure on page 3 "T cells vs. antibodies" (T cells curve) appears to represent "N or S or M" so it should… https://t.co/TTKMvRwUxk
Even stranger: the authors seem to have mixed up the data sources of the mini figure by building the first two poin… https://t.co/aJr4Qv5tBk
This "error" looks like data massaging to make T cells look higher than Antibodies on the mini figure. Suspicious. 8/n
In fact all these figures (3A, 4G, mini figure) conflict with each other one way or another. It is mathematically… https://t.co/KrSg4JcpLl
All these self-inconsistencies mean readers truly have no idea which is the correct percentage of individuals elici… https://t.co/s8Zp2xOEd3
We find a clue of a possible explanation behind these numerous errors in the legends of figures 1 through 4... each… https://t.co/4BaUkYT7Qe
The legend of figure 1 states the charts were drawn using: - 18 of the 55 2020_BD cases - 11 of the 10 AM cases (wh… https://t.co/REwm0TzoCm
And figure 2 charts were drawn using: - 7 of the 55 2020_BD cases - 11 of the 40 MC cases - 6 of the 17 AS cases 1… https://t.co/4sdOe8J7i8
And figure 3 charts were drawn using: - 25 of the 28 2019_BD cases - 24 of the 55 2020_BD cases - all 30 Exp cases… https://t.co/SneuBbE9Hf
And figure 4 charts were drawn using: - 30 of the 28 2019_BD cases (an error, 30>28) - 31 of the 55 2020_BD cases -… https://t.co/BJswWEvsAc
It is suspicious for the authors to selectively ignore large fractions of cases in different parts of the analysis… https://t.co/3akddJzcIz
For example, from figure 3A (left part, N or S or M) showing 2020_BD about twice as high as 2019_BD, the authors de… https://t.co/awRgE9mZGB
«about twice as many healthy individuals who donated blood during the pandemic generated memory T cell responses [.… https://t.co/yq4q7JjKsZ
However figure 3A ignores most 2020_BD cases: it charts 24, and ignores 31, of the 55 cases. If the 31 that they ig… https://t.co/vuh3pAAA2K
...between 2019_BD & 2020_BD cases, and the theory that immunity may be underestimated is no longer supported! Sel… https://t.co/elVYvoW818
On a more minor note, in various places the paper references supplementary figures S1 through S5. However supplemen… https://t.co/iNyIVY0LZj
Bottom line, the numerous errors and inconsistencies in the data at the center of this paper mean we cannot trust b… https://t.co/3Cg2Lrg965
It is probable all these errors are merely caused by the authors failing to update the figures as they accumulated… https://t.co/EFSX96Z1OL
No matter what is the explanation, even more concerning is: why did peer review detect none of these errors? Absolu… https://t.co/NIagpRcLoZ
Thank you the Scientific Reviewer for going through this. Most of the numbers and data you are referring to as “inconsistency” and “suspicious” is based on the fact that we were using two different assays to assess the T cell response. The first data set in figure 3 is ELISPOT and figure 4 is proliferation assays. Any one actually working with T cells knows that proliferation assays are more sensitive and therefore generates somewhat higher response rates than conventional ex vivo assays. These reviewers are experts in the field and knows this and therefore did not point out the differences in response frequencies. For the general public of course, it could be confusing using two different assays to display numbers - I acknowledge that - but in the end, we decided to fully display all data that was generated using both ELISPOT and proliferation assays. Let me know if you have any more concerns about the paper and I’m happy to directly interact with you.
/Marcus
Thank you, this explains one inconsistency. Can you respond to the other errors and our comments?:
This claim is false: "about twice as many healthy individuals who donated blood during the pandemic generated memory T cell responses in the absence of detectable circulating antibody responses". As noted, antibodies are detectable in 4 out of 31 2020_BD cases (fig 4G). When removing the 4 cases with antibodies, the difference is no longer "twice as many".
It is misleading for the "t cells vs. antibodies" minifigure T cell curve to represent the highest of either the ELISPOT or proliferation assays. This is cherry picking.
The study is introduced as having a large number of 2020_BD cases (n=55), but none of the analyses or figures show data for the 55 cases. Figures 1 through 4 analyze respectively 18, 7, 24, 31 cases. This poor methodology enables cherry picking. If there were not enough resources to test all 55 cases, you should have started with fewer than 55 samples, and run all assays/tests on the same samples.
Figure 4's legend claiming 30 2019_BD cases, when there are only 28.
Figure 1's legend claiming 11 AM cases, when there are only 10.
Conflicting claims of 61% vs 64% of Exp cases having antibodies. "64%" of 30 Exp cases is probably a typo (19 of 30 would be 63%, not 64%).
Thank you for the response and pointing out some of the obvious errors in the text!
What we mean with this statement is that twice as many generate a broad T cell response compared to antibodies. And be aware here, that there might be more people that for instance generates a response against one antigen (such as the spike region) that we usually see in the unexposed donors. We here only took into consideration the people that recognized N and M or S when we make this statement. We could easily then have said that we see even many times more (unpublished observations) donors with T cells recognizing the virus - but we didn’t to not oversell our data. We can try to rephrase this when we get the final proofs.
Same as above: If we would cherry-pick data that made our data look even “better”, then we could have easily picked all patients generating a T cell response against any antigen (S, N or M). But we did not in this illustration. Furthermore, this is a graphical abstract and should therefore illustrate our findings. One could then also comment on the cells next to the graph and question whether all the TNF molecules really goes in line with the data. The point is that this is a graphical abstract and not raw data and we tried to emphasize that we generally in our assays see more T cell than antibody responses in the different groups. Same here as above, could be changed when we get the proof if too many thinks this is confusing.
We have not “cherry-picked“ anything - this is a very strong wording. We have access to a limited amount of cells per donor that doesn’t allow us to do all analysis. In addition, we had to share with multiple other groups PBMCs from all of these donors for conducting all the analysis. If this is poor methodology, then you should look into all animal studies where people sacrifice animals and use new batches of animals for all type of analysis - they can’t reuse the same animals again. As pre-clinical scientists, we try to do as many analysis as possible based on what we, collaborators and reviewers ask for - therefore there will be different number of subjects in some of the analysis. I can point you towards multiple other publications if you want to see this for yourself!
This is a typo from our side. We will change this in the final version - after we get the proofs. Thanks for pointing this out.
This is a typo from our side. We will change this in the final version. Thanks for pointing this out.
61% of the exposed subjects that went through the proliferation assay had antibodies - total of 28 subjected. In total, we recruited 30 subjects, where you are right that 63.3% exactly had antibodies.
Let me know if you want to have a proper direct discussion also about the quality of the work (flow data etc) for instance in comparison to other published work out there and the number of analysis that we were able to do with this material! Also let me know if you have any more issues to directly discuss with me Ventrifrossa.
1a (new point). That said, the "twice as many" comparison between 2019_BD and 2020_BD cases should exclude the 4 2020_BD seropositive cases. These individuals have been exposed to SARS-CoV-2 as they have detectable antibodies, therefore T cell responses are obviously expected. A proper and more interesting comparison would be: if we exclude these 4 individuals do we still measure more T cell responses in 2020_BD compared to 2019_BD? It is not clear at all in your data. In figures 4B and 4C, if we assume the 4 seropositive cases are among the dots showing the highest reactivity, if we exclude 4 of these dots we don't see a significant difference between 2019_BD and 2020_BD (except perhaps in 1 of the 6 charts: figure 4B, CD4+, Nucleocapsid reactivity.)
We can accept more freedom was taken to draw an artistic representation of TNF molecules, but in our opinion more rigor should be employed when charting precise points along a y-axis, especially in a journal like Cell.
Ok, we understand your limitations due the limited amount of cells. In our opinion the study should mention this limitation.
4, 5, 6: Ok. We look forward to the corrections.
7 (new point). What's the timing between the moment Exp individuals were exposed, and the moment blood samples were collected? This could be an important limitation of the study. The sampling timing is documented for all groups, except the Exp group. If samples were collected shortly after exposure, then it may not have given enough time for their immune system to develop antibodies.
Thank you for taking the time to engage.
1a) Agree that this could have been an interesting comparison to do. We are currently working on a larger blood donor cohort and will keep these analysis in our minds. Thank you.
Agree that it is important to be rigor and maybe more importantly conduct research of high quality when a paper is published in higher tier-journals. I could easily spend my entire time to point out questionable flow and sequencing data as well as other things I have seen in many COVID-19 papers coming out in prominent journals. But, I chose instead to spend my time on conducting innovative work - and even that is not always enough to capture minor things that you pointed out with frequencies in this paper.
Agree, we will see if we could, and are allowed, to emphasize this in the final proof.
4-6) Again, thanks for pointing these things out.
Thank you for your questions! And as I wrote above- better that you email me directly with the questions if you have any more of them.
Therefore 67% more exhibit T cell responses compared to those with antibodies.
1a-6: Ok
We have no further comments or questions. If we do, we will contact you by email from now on. Thanks again for taking the feedback. We look forward to the revisions, and other studies in progress from your team.
Attach files by dragging & dropping, selecting them, or pasting from the clipboard. Uploading your files… We don’t support that file type. with a PNG, GIF, or JPG. Yowza, that’s a big file. with a file smaller than 1MB. This file is empty. with a file that’s not empty. Something went really wrong, and we can’t process that file.
Comment must be at least 15 characters.