Notices
Results 1 to 66 of 66
Like Tree4Likes
  • 1 Post By Paleoichneum
  • 1 Post By John Galt
  • 1 Post By Lynx_Fox
  • 1 Post By John Galt

Thread: Rejected: Peer Reviewers on Strike

  1. #1 Rejected: Peer Reviewers on Strike 
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Peer review is the evaluation of work by one or more people of similar competence to the producers of the work (peers.) It constitutes a form of self-regulation by qualified members of a profession within the relevant field (from wikipedia).

    So far, so good. However, peer-reviewing, in many scientific professions, is a free service that is mostly performed by scientists on short-running positions, who do not have any time to waste on their peer's garbage. This renders the peer-reviewing process to have very low quality, and as a result a large proportion of scientific articles are not worth reading.

    We want to help fixing this system. We demand a comprehensive scheme of compensation for peer reviewing. Quality control must not be an act of charity: it needs to be recognized as a valuable part in the academic process.

    Join the process.


    Reply With Quote  
     

  2.  
     

  3. #2  
    Moderator Moderator
    Join Date
    Apr 2007
    Location
    Washington State
    Posts
    8,322
    I've never run into the attitude that you describe, but admittedly my experiences were limited to climate, meteorology and the military testing community.


    Meteorologist/Naturalist & Retired Soldier
    “The Holy Land is everywhere” Black Elk
    Reply With Quote  
     

  4. #3  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Mayhap, my formulations were generated under some kind of frustration. However, I would be very glad to hear other opinions.
    Reply With Quote  
     

  5. #4  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    i can say from my experiance with paleobotany and paleoentomology articles, that has not been the case.
    Neverfly likes this.
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  6. #5  
    Suspended
    Join Date
    Nov 2012
    Location
    Apocalyptic Paradise
    Posts
    6,613
    I hope others that directly work with peer review chime in. I admit, the O.P. didn't seem like a bad suggestion. There are more than one type of reader out there and while I doubt there's an intentional bias... I think that there are times when peers are tired, get a knee jerk reaction or feel overwhelmed and under-appreciated.
    Just because I see that as possible, doesn't mean that is how it is considering many in the field take their work very seriously- I've worked with too many that do and they are amazing in how they maintain the science at all times and seem tireless.
    Reply With Quote  
     

  7. #6  
    Universal Mind John Galt's Avatar
    Join Date
    Jul 2005
    Posts
    14,169
    Quote Originally Posted by Kevin Clever View Post
    However, peer-reviewing, in many scientific professions, is a free service that is mostly performed by scientists on short-running positions, who do not have any time to waste on their peer's garbage. This renders the peer-reviewing process to have very low quality, and as a result a large proportion of scientific articles are not worth reading.
    This has the appearance of an opinion. I would oppose the formulation of a radically different strategy based upon an opinion. Do you have - and I understand the irony - peer reviewed research that would tend to support your assertion?
    Reply With Quote  
     

  8. #7  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    There's an urban myth that the same research gets repeatedly published every 15 years. For nonlinear dynamics I cannot dismiss this hypothesis upon my experience. To give anecdoctal evidence for another subject (largely for amusement), I read once in a journal of medicine an article from the 90th describing numerical differentiation as a new tool.

    What I am wondering is, however, why you oppose the notion of reembursing diligent work so much? I know that the science community is prone to be politically on the left side of the river. This is typically ok for me, however, it does not reflect the hard realities that you face when living at the age of 30 on a two year position. The system is in fact very right and market liberal. In fact, the liberal market forces have reached the conclusion that one doesn't even need to pay scientists to make them work as peer-reviewers for example, and now you are telling me that this should exactly be this way?

    What I would like to see is an international labour union that assures that a few things, such as underpaid peer-revierwing, do not remain common practice.
    Other interesting effects could be:

    (1) Scientific results generated by researchers like me (yes I care for my own contribution) shall not be abused for sinister purposes.
    (Ettore Majorana, a nuclear theorist who may have disappeared because it became clear to him that his research would lead to nuclear weapons, would probably agree with me: http://iopscience.iop.org/1742-6596/...173_012019.pdf Also, you don't seem to hear much of him in these regards.)

    (2) Scientists receive their "fair share" in the generation of revenue. Some of the new technology is based upon research from the past, yet this fact seems to be largely ignored by patent law.
    Reply With Quote  
     

  9. #8  
    Genius Duck Moderator Dywyddyr's Avatar
    Join Date
    Jan 2013
    Location
    Scunthorpe, UK
    Posts
    10,824
    Quote Originally Posted by Kevin Clever View Post
    Ettore Majorana, a nuclear theorist who may have disappeared because it became clear to him that his research would lead to nuclear weapons
    Yeah, that's likely.
    Considering Szilard had filed for patent on neutron-induced chain reactions (you know, the sort that lead to weapons) FIVE years earlier...
    Last edited by Dywyddyr; March 6th, 2013 at 04:29 PM.
    "[Dywyddyr] makes a grumpy bastard like me seem like a happy go lucky scamp" - PhDemon
    Reply With Quote  
     

  10. #9  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    Kevin, Who would do the reimbursing and from what funds? I can tell you that there is NO money in the vast majority of paleontology fields.
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  11. #10  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Paleoichneum View Post
    Kevin, Who would do the reimbursing and from what funds? I can tell you that there is NO money in the vast majority of paleontology fields.
    Dear Paleoichneum,

    I cannot talk for the field of paleontology. However, it's in the best interest
    of your institution to pay for submissions. They don't do that because it is
    not common practice.

    In the model that I think of currently, there is not even payment involved.
    A symbolic amount as a gesture of respect, and a "winner's point" of
    reputation at an independent counting agency to keep track of who
    does what amount of work would be appropriate, I think, but this is
    really open to discussion. Self-evolving this could be directly linked
    to the amount that you need to pay in your future submissions. I don't
    know, but some smart economist can probably come up with a fun
    scheme to give the right incentives in these questions.

    Something like that would also help me explaining to my wife
    what-the-hell I did late at night at office.

    As it is now, you wouldn't even be able to proof to a prospective employer
    that you have reviewing abilities recognized among editors in your field of
    expertise.

    What bothers me a little bit is the fact that no one else in this block
    seems to think that this is a good idea, and something new could be
    refreshing for the science community.

    K
    Reply With Quote  
     

  12. #11  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    What experiance have you had with peer-review and in what fields? Do you more often submit or review?
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  13. #12  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    I seem to be registered as a peer-reviewer in physics journals, for which
    I more often read than write. I suppose that this is usual in physics where
    pressure for publication is high as many want to stay in science. The amount of
    reviewing seems to depend on how quickly and well you do your "homework".

    How about you?

    Btw.: Today I heard that peer-reviewing for American journals will
    give you a preference to receive a green card in the future. This is
    however not very helpful for those who were born in the USA
    because they hardly need one.
    Reply With Quote  
     

  14. #13  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    What do you mean "I seem"?

    Where did you hear that from?

    In the paleobotany area of peer-review its a volunteer position, not a requirement.
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  15. #14  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    I volunteer in cleaning the Charles River. Every month, I wipe my arse out
    at 5 am to go to the river and sample some water. You see that I very much
    understand the process of volunteering. However, I also came to notice
    that society uses our willingness to do free services where ever they can.

    To me, this is ok as long as you are not giving away your actual profession
    for free. No one does that except musicians and artists, and we shouldn't
    either. I want scientists to strive and to take a leading role in society, and
    thus one has to act like a serious person that him- or herself believes in
    the value of one's everyday work.
    Last edited by Kevin Clever; March 7th, 2013 at 10:08 AM.
    Reply With Quote  
     

  16. #15  
    Life-Size Nanoputian Flick Montana's Avatar
    Join Date
    Jul 2012
    Location
    Flatland
    Posts
    5,438
    Redacted

    I, for one, would have reservations about monetizing the peer review process.
    Last edited by Flick Montana; March 7th, 2013 at 11:57 AM.
    "Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us." -Calvin
    Reply With Quote  
     

  17. #16  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Flick Montana View Post
    Quote Originally Posted by Kevin Clever View Post
    You need to know, Paleoichneum, that your method of questioning is a
    little bit impolite. While other's will simply ignore you after a while, I
    wanted to tell you this in case you didn't notice.
    On the contrary, I think you're the one being a bit standoffish. Paleo has been one of the most polite and helpful people I've encountered on this forum.

    I, for one, would have reservations about monetizing the peer review process.

    I would be curious to hear what are the problems that you see? It could help greatly
    in designing solutions that everyone can be happy with.

    As to the comment that I have posted earlier, and especially about the part that
    I decided to delete afterwards because I found it inappropriate altogether, my
    critique circles around the habbit of asking, but not answering questions, and also
    about the general habbit of trying to find out about other peoples opinion without
    giving away one's own. While this may seem to some as a very smart approach
    to converse, it was perceived by me as impolite.
    Reply With Quote  
     

  18. #17  
    Life-Size Nanoputian Flick Montana's Avatar
    Join Date
    Jul 2012
    Location
    Flatland
    Posts
    5,438
    Quote Originally Posted by Kevin Clever View Post
    I would be curious to hear what are the problems that you see? It could help greatly
    in designing solutions that everyone can be happy with.
    My primary issue with monetizing the process is that I have rarely seen the injection of potential profits into a process create a positive outcome. I also have concerns in the peer review process when it comes to conflicts of interest. I don't think this could be corrected through a paid compensation.

    I have started to take interest in an open and dynamic peer review process over typical anonymous submissions. I feel accountability would discourage fallacy. Of course, that process has its limitations as well.

    Honestly, if I knew a solution or if I were 100% behind a certain method, I'd gladly throw it out there, but I'm not completely sold on the entirety of anything I've seen proposed. I'm not sure there really is a silver bullet to the peer review criticisms out there.

    Also, I didn't realize you deleted part of your post. I did the same out of respect.
    "Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us." -Calvin
    Reply With Quote  
     

  19. #18  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Do you mean a "cites you like" approach? I think this could be very interesting,
    where publishing is essentially open as in arXiv, but the community rates as it
    reads. This is a very senseble approach, however, does it take of much? In my
    field, the head count is peer-reviewed publications/H-index and such.

    While not having spent much time with it, cites-u-like seems to have a very
    unnatural interface. Has anybody better open web resources of crowd rating?

    One potential pitfall is additional hurdles for new comers to enter a field, I guess.
    Reply With Quote  
     

  20. #19  
    Life-Size Nanoputian Flick Montana's Avatar
    Join Date
    Jul 2012
    Location
    Flatland
    Posts
    5,438
    Sort of like the Philica online journal. Review takes place after publication. It's an interesting experiment, but I still have my concerns about it. One thing I do very much enjoy is the ability of the publishers to openly defend their submission immediately and during the review process.
    "Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us." -Calvin
    Reply With Quote  
     

  21. #20  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    My primary issue with monetizing the process is that I have rarely seen the injection of potential profits into a process create a positive outcome. I also have concerns in the peer review process when it comes to conflicts of interest. I don't think this could be corrected through a paid compensation.

    Also, I didn't know that you were a bigger system criticist
    than I am. However, we are not to forget that we live in
    a money based economy.

    Your doctor may admit that he thinks that you are a very
    nice guy in that you really help science by being a good
    peer-reviewer, but he will also say that this simply will not
    allow you to take a free ride in the MRI.

    In the end, we are the helpful idiots of society. That
    there are people who think money is not the answer is,
    by the way, very helpful to the establishment.
    Reply With Quote  
     

  22. #21  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    ... while others have parties with prostitutes and cocaine,
    scientists can't sleep over worries about next year's grant.
    Reply With Quote  
     

  23. #22  
    Ascended Member Ascended's Avatar
    Join Date
    Apr 2012
    Location
    UK
    Posts
    3,380
    Do any peer reviewers actually currently get paid for the time or efforts and if so how much and where?
    Everything has its beauty, but not everyone sees it. - confucius
    Reply With Quote  
     

  24. #23  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Ascended View Post
    Do any peer reviewers actually currently get paid for the time or efforts and if so how much and where?
    Most reviewers of grant application panels are paid to my knowledge.
    I would assume that you are also paid to review Nature articles, but I
    don't know.
    Reply With Quote  
     

  25. #24  
    Moderator Moderator
    Join Date
    Apr 2007
    Location
    Washington State
    Posts
    8,322
    Science is more about intangibles as it is about money--heck most in their right mind because scientist with becoming rich on their minds.

    If anything peer review might well hurt sciences because it will create one more barrier to free sharing of information since the money has to come from somewhere.

    Is there any empirical evidence of problems of the peer review system being created by non-paid peer reviews?
    Meteorologist/Naturalist & Retired Soldier
    “The Holy Land is everywhere” Black Elk
    Reply With Quote  
     

  26. #25  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Lynx_Fox View Post
    Science is more about intangibles as it is about money--heck most in their right mind because scientist with becoming rich on their minds.

    If anything peer review might well hurt sciences because it will create one more barrier to free sharing of information since the money has to come from somewhere.

    Is there any empirical evidence of problems of the peer review system being created by non-paid peer reviews?
    Obviously the problem of non-paid peer-reviewing is insignificant.
    Somehow, you also miss the point, so I will try an anecdote:

    When I was in college I had two roommates. Because I didn't have
    much time, all the cleaning was done by the roommates. One
    day, one of the roommates noticed that I never help to clean up
    and decided that he will do so as well. Up from that day, all the
    cleaning was done by one of the roommates while me and the
    other enjoyed frequent outings to local pubs and bars.

    Moral: I don't want to be the one roomate cleaning everything by
    herself, even if I think cleaning is relaxing and I do it very well.

    (next time, I will make up a fable with the cunning fox and the rabbit)
    Reply With Quote  
     

  27. #26  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    Im confused, are you suggesting that researchers are stopping peer-reviewing?
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  28. #27  
    Universal Mind John Galt's Avatar
    Join Date
    Jul 2005
    Posts
    14,169
    I find discussions on discussion forums work best when discussion takes place. One aspect of discussion is to respond to questions. Perhaps you missed this:

    You said - "as a result a large proportion of scientific articles are not worth reading."

    I replied "This has the appearance of an opinion. I would oppose the formulation of a radically different strategy based upon an opinion. Do you have - and I understand the irony - peer reviewed research that would tend to support your assertion? "

    Any chance of a response?
    Halliday likes this.
    Reply With Quote  
     

  29. #28  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Paleoichneum View Post
    Im confused, are you suggesting that researchers are stopping peer-reviewing?
    Yes, it's called industrial action. I'm sure journals would react quickly.
    One could for example still post @ online/open journals, so
    in fact I don't think that it would even effect the scientific everyday
    business very much.

    Sometimes the military steps in these cases, but I highly doubt that
    this will happen.
    Reply With Quote  
     

  30. #29  
    Moderator Moderator
    Join Date
    Apr 2007
    Location
    Washington State
    Posts
    8,322
    Kevin, please join the conversation.

    You are making some pretty wild direct and inferred claims about the state of peer review, yet seem to be completely avoiding attempts to provide evidence of any problem with the current system--peer review studies of the problem, not even a direct anecdote.

    Science publications remain at a very high rate, in fact they continue to climb, so there doesn't appear to be a serious problem of difficulty finding reviewers. Your statement that articles aren't worth reading also needs support and as John pointed out, does seem to be more opinion than anything else.
    Meteorologist/Naturalist & Retired Soldier
    “The Holy Land is everywhere” Black Elk
    Reply With Quote  
     

  31. #30  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Lynx_Fox View Post
    Kevin, please join the conversation.

    You are making some pretty wild direct and inferred claims about the state of peer review, yet seem to be completely avoiding attempts to provide evidence of any problem with the current system--peer review studies of the problem, not even a direct anecdote.

    Science publications remain at a very high rate, in fact they continue to climb, so there doesn't appear to be a serious problem of difficulty finding reviewers. Your statement that articles aren't worth reading also needs support and as John pointed out, does seem to be more opinion than anything else.
    To the point, the greatest failure of scientific peer-reviewing
    is not to re-emburse the peer-reviewer.




    There's more to say though:

    The last paper that I submitted took 1 month to be reviewed. The response
    was one paragraph long and contained a concern that was actually covered
    in that draft. I can't blame this guy because he/she probably has other things
    to do. (We are by the way in the process of responding.)

    Another reviewer came up with 2 pages containing 20 well-explained points
    of critique that helped tremendously to make the article most clearly readable.

    Sometimes, there is also just a complain that one should cite this and that
    person, raising my suspicion that some reviewers want to increase their
    own citation index. This is by the way the only incentive that I can currently
    think of (besides altruistic incentives of which I deny any existence).

    What I get out of this is, that peer-reviewers can be very helpful in creating
    a well-rounded and good manuscript, but it can also be frustrating to the
    authors because of peer-reviewers who simply do not take any time.
    Reply With Quote  
     

  32. #31  
    Life-Size Nanoputian Flick Montana's Avatar
    Join Date
    Jul 2012
    Location
    Flatland
    Posts
    5,438
    Quote Originally Posted by Kevin Clever View Post
    However, we are not to forget that we live in
    a money based economy.
    I'm finding myself coming back to this comment.

    While I agree that we need money in the sciences for funding and support (quite a lot of money, actually), I don't feel as though many people in the sciences are driven by money. Granted, I had a couple friends who went off to make 6 figures doing geological work for Big Oil, but for the most part, I think we're in this business because we can't help it.

    While a paycheck is nice (a bigger one wouldn't hurt), I get infinitely more satisfaction from my work as a problem solver. I don't know if offering compensation for review would bolster the scientists who peer review or if it would attract unscrupulous practices. When I say I don't know, I really mean it. I have no clue what would happen, but it makes me slightly uncomfortable to think about it for some reason.
    "Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us." -Calvin
    Reply With Quote  
     

  33. #32  
    Moderator Moderator
    Join Date
    Apr 2007
    Location
    Washington State
    Posts
    8,322
    My own experiences were similar--some were excellent, some weren't, others were just formalizations of points I'd already heard at conferences or in correspondence which sometimes made it easy to identify the "ananymous" reviewer. I'm not sure paying them would change any of this--even attempts to formalize and make paying reviewers accountable might well fail. Ultimately this is why many editors often get at least three reviews--those same editors also get a sensing of who's doing a good job (regardless of pay) and has an insentive to continue asking them to review to maintain the credibility of the journal. All I really see improving from a paid review system is perhaps more timeliness.

    This is by the way the only incentive that I can currently
    think of (besides altruistic incentives of which I deny any existence).
    Why? Don't you think scientist want to contribute to their own fields? Don't think they have a sense of professionalism? Just being a reviewer brings some credibility as well. The reviewers opportunities to see work months before others in their field is another substantial benefit.

    I'm not married to either system. But I'm not convinces it's broken either and needs any fixing. I'm a lot more worried about continued lack of affordable availability of research to other fields of scientist and the general public. I think paid reviewers would make information and research even less available.
    Flick Montana likes this.
    Meteorologist/Naturalist & Retired Soldier
    “The Holy Land is everywhere” Black Elk
    Reply With Quote  
     

  34. #33  
    Life-Size Nanoputian Flick Montana's Avatar
    Join Date
    Jul 2012
    Location
    Flatland
    Posts
    5,438
    Quote Originally Posted by Lynx_Fox View Post
    Why? Don't you think scientist want to contribute to their own fields?
    I think this is the nail on the head.

    If you ask a scientist what is most important to them, their answer will likely be 'Integrity'. If you can help to maintain the integrity of your field through peer review, you will do it. I don't know a scientist who wouldn't. Regardless of compensation.
    "Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us." -Calvin
    Reply With Quote  
     

  35. #34  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Another possibility would be to drop the review
    process alltogether, and to base relevance
    of scientific articles on citation. However, I do
    not know how a transition to a system without
    ranking in scientific journals would work.
    Reply With Quote  
     

  36. #35  
    Life-Size Nanoputian Flick Montana's Avatar
    Join Date
    Jul 2012
    Location
    Flatland
    Posts
    5,438
    Quote Originally Posted by unigrad View Post
    I don't have much to add. When money is involved, all sorts of people come out of the wood work. Greed is a vicious thing.
    The irony of your sig is not lost on me.
    "Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us." -Calvin
    Reply With Quote  
     

  37. #36  
    Ascended Member Ascended's Avatar
    Join Date
    Apr 2012
    Location
    UK
    Posts
    3,380
    Quote Originally Posted by unigrad View Post
    I don't have much to add. When money is involved, all sorts of people come out of the wood work. Greed is a vicious thing.
    I'm just wondering here though guys if perhaps Kevin's idea might have some more merit at an early stage, perhaps relevently qualified individuals could agree to give an assesment prior to a full peer review process, to help iron out wrinkles, for a fixed level of compensation for their time and effort. This may also give some of the science community the opportunity to earn some extra money if they want to. This could provide a useful service without really interfering to much in the impartiality of the full independent process.
    Everything has its beauty, but not everyone sees it. - confucius
    Reply With Quote  
     

  38. #37  
    Suspended
    Join Date
    Nov 2012
    Location
    Apocalyptic Paradise
    Posts
    6,613
    Quote Originally Posted by Ascended View Post
    I'm just wondering here though guys if perhaps Kevin's idea might have some more merit at an early stage, perhaps relevently qualified individuals could agree to give an assesment prior to a full peer review process, to help iron out wrinkles, for a fixed level of compensation for their time and effort. This may also give some of the science community the opportunity to earn some extra money if they want to. This could provide a useful service without really interfering to much in the impartiality of the full independent process.
    I think the idea has been poorly presented. But I agree with you, I wonder the same thing. I agree that some folks are in it for the duty- but not everyone is.
    Reply With Quote  
     

  39. #38  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Lynx_Fox View Post
    Why? Don't you think scientist want to contribute to their own fields? Don't think they have a sense of professionalism? Just being a reviewer brings some credibility as well. The reviewers opportunities to see work months before others in their field is another substantial benefit.
    "No, middle income is $200,000 to $250,000 and less," -- Mitt Romney

    Not everybody can afford that attitude.
    Reply With Quote  
     

  40. #39  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    It still begs the question of where the money would come from. I am also highly skeptical of the continued assertion that there is a lack of reviewers and so money is needed to provide incentive.

    What would doing away with reviewing do? Reviewing is an important part of the process for streamlining papers and catching errors in data/methodology/conclusions/grammar etc before publication.
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  41. #40  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Paleoichneum View Post
    It still begs the question of where the money would come from. I am also highly skeptical of the continued assertion that there is a lack of reviewers and so money is needed to provide incentive.

    What would doing away with reviewing do? Reviewing is an important part of the process for streamlining papers and catching errors in data/methodology/conclusions/grammar etc before publication.
    This is really a matter of implimentation.

    One PPR model (paid-peer reviewing) would be based on a submission
    fee. As Lynx_Fox has pointed out, it doesn't actually take much money.

    One way to think of it would be the following: An average research that
    publishes (as co-author partitioned) 2 papers per year should not pay more
    than $100 as submission fee. With two reviewers per article, this would
    give each of the reviewers $25, which is enough to spend 2 hours of
    work (considering some kind of minimum wage) of concentrated work
    with the manuscript.

    The journal on the other side could implement a que where the editor is
    given authority of whether the money is transfered to the reviewer or not.

    The incentive of the editor is to uplift the impact of the journal, and the
    incentive of the reviewer would be to please the editor (not the author;
    That's important).

    My experience is that a thoroughly checked review, including references,
    demands for more than 2 hours. However, if the article fails within the
    first two hours to draw on the reviewer's interest to invest his own research
    efforts she will probably be able to explain why, and the response should
    be out within two days. If it takes longer, then editor may expect a more
    thorought response of the reviewer. Things could be more transparent
    in this sense.
    Reply With Quote  
     

  42. #41  
    Moderator Moderator
    Join Date
    Apr 2007
    Location
    Washington State
    Posts
    8,322
    Quote Originally Posted by Kevin Clever View Post
    Quote Originally Posted by Lynx_Fox View Post
    Why? Don't you think scientist want to contribute to their own fields? Don't think they have a sense of professionalism? Just being a reviewer brings some credibility as well. The reviewers opportunities to see work months before others in their field is another substantial benefit.
    "No, middle income is $200,000 to $250,000 and less," -- Mitt Romney

    Not everybody can afford that attitude.
    Romney's opinion has nothing to with this thread.

    That being said, scientist for the most part are not hurting for money--particularly those with reputations and contributions significant enough to become a reviewer.

    You'd make a stronger case by actually presenting evidence that supports your opinion, rather than trying to politically polarize the thread.
    Meteorologist/Naturalist & Retired Soldier
    “The Holy Land is everywhere” Black Elk
    Reply With Quote  
     

  43. #42  
    Ascended Member Ascended's Avatar
    Join Date
    Apr 2012
    Location
    UK
    Posts
    3,380
    Quote Originally Posted by Kevin Clever View Post

    One PPR model (paid-peer reviewing) would be based on a submission
    fee. As Lynx_Fox has pointed out, it doesn't actually take much money.

    One way to think of it would be the following: An average research that
    publishes (as co-author partitioned) 2 papers per year should not pay more
    than $100 as submission fee. With two reviewers per article, this would
    give each of the reviewers $25, which is enough to spend 2 hours of
    work (considering some kind of minimum wage) of concentrated work
    with the manuscript.
    But it seems here you are still financially linking the authors with the reviewers, which is always going to raise questions about impartiality, also opening up problems if the authors don't like a review, since they are effectively paying for this service.
    Everything has its beauty, but not everyone sees it. - confucius
    Reply With Quote  
     

  44. #43  
    Ascended Member Ascended's Avatar
    Join Date
    Apr 2012
    Location
    UK
    Posts
    3,380
    Quote Originally Posted by Neverfly View Post
    I think the idea has been poorly presented. But I agree with you, I wonder the same thing. I agree that some folks are in it for the duty- but not everyone is.
    Yeah it seems fair that if people are spending their time helping to review the work of others then why shouldn't they be able to at least cover the financial costs of their time, but it's doing it in an impartial way that doesn't make it to expensive.
    Everything has its beauty, but not everyone sees it. - confucius
    Reply With Quote  
     

  45. #44  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Lynx_Fox View Post
    Quote Originally Posted by Kevin Clever View Post
    Quote Originally Posted by Lynx_Fox View Post
    Why? Don't you think scientist want to contribute to their own fields? Don't think they have a sense of professionalism? Just being a reviewer brings some credibility as well. The reviewers opportunities to see work months before others in their field is another substantial benefit.
    "No, middle income is $200,000 to $250,000 and less," -- Mitt Romney

    Not everybody can afford that attitude.
    Romney's opinion has nothing to with this thread.

    That being said, scientist for the most part are not hurting for money--particularly those with reputations and contributions significant enough to become a reviewer.

    You'd make a stronger case by actually presenting evidence that supports your opinion, rather than trying to politically polarize the thread.

    You are right that I would make a stronger case, but unfortunately
    I don't know any journal that pays its reviewers.

    Furthermore, the first statement is not true. I did not get paid for most
    part of last year, while still answering requests from EPL. I was applying
    for jobs.

    Also sometimes Phd-students are transfered reviewing tasks
    for reasons which I don't understand.
    Reply With Quote  
     

  46. #45  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Ascended View Post
    Quote Originally Posted by unigrad View Post
    I don't have much to add. When money is involved, all sorts of people come out of the wood work. Greed is a vicious thing.
    I'm just wondering here though guys if perhaps Kevin's idea might have some more merit at an early stage, perhaps relevently qualified individuals could agree to give an assesment prior to a full peer review process, to help iron out wrinkles, for a fixed level of compensation for their time and effort. This may also give some of the science community the opportunity to earn some extra money if they want to. This could provide a useful service without really interfering to much in the impartiality of the full independent process.
    Do you mean to find a private company? It exists actually
    for grant applications, especially in france where people
    are prone not to speak very good English. However, here
    the emphasis is writing and presentation, not science.
    Reply With Quote  
     

  47. #46  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Concerning deleted post about linking author and reviewer (?):


    Yes, this is true, but the issue is similar to the
    one that teachers have who make students
    take test. I assume that some lawyer and
    some economist can sit together for a day
    and solve the implementation problem
    thoroughly...
    Reply With Quote  
     

  48. #47  
    Moderator Moderator
    Join Date
    Apr 2007
    Location
    Washington State
    Posts
    8,322
    What implementation problem? You haven't shown there is one. Until you establish there is one, there's not much point in recommending solutions.
    Meteorologist/Naturalist & Retired Soldier
    “The Holy Land is everywhere” Black Elk
    Reply With Quote  
     

  49. #48  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by Lynx_Fox View Post
    What implementation problem? You haven't shown there is one. Until you establish there is one, there's not much point in recommending solutions.
    My comment was refering to a deleted post. I am sorry about that.
    Reply With Quote  
     

  50. #49  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Also, I think this conversation has reached a natural end
    because most arguements and opinions seem to be posted
    already. It was enlighting to discuss with you, and to receive
    some oppinion on the pros and cons.

    IF you want to be part in this, please join the equally-named
    group on facebook. Together we are strong.

    I'm sorry for all the insulting comments. I guess I should
    cool down, or finally go the dentist or so.
    Reply With Quote  
     

  51. #50  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    So we are not actually getting to see your data and methods section of your thesis, just an abstract? Hmmmm, not so impressive.
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  52. #51  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Nobody reads these parts. Not even the peer-reviewer.
    Reply With Quote  
     

  53. #52  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    Actually the data and materials sections are the most important parts of the papers. Have you linked this discussion thread to your Facebook group so the other 121 members may see this and participate?
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  54. #53  
    Universal Mind John Galt's Avatar
    Join Date
    Jul 2005
    Posts
    14,169
    Quote Originally Posted by Kevin Clever View Post
    Also, I think this conversation has reached a natural end
    because most arguements and opinions seem to be posted
    already.
    May I ask you for a third time to response to my query?

    You said - "as a result a large proportion of scientific articles are not worth reading."

    I replied "This has the appearance of an opinion. I would oppose the formulation of a radically different strategy based upon an opinion. Do you have - and I understand the irony - peer reviewed research that would tend to support your assertion? "

    Any chance of a response?

    Please don't offer an apology for your oversight. Just answer the question.
    Lynx_Fox likes this.
    Reply With Quote  
     

  55. #54  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Sorry 'bout that. Here's an interesting reference
    supporting some of my views. While the author,
    by the way far more experienced than me, seems
    to have given up on the subject, I do not like his
    conclusions. We need to think hard to finally fix
    the system, and I believe a joint effort will bring
    drastic change to 21st century publishing.

    Peer review: a flawed process at the heart of science and journals
    Reply With Quote  
     

  56. #55  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    I would like to make another interesting point,
    regarding another article I had read.

    » An economic solution to reviewing load The Occasional Pamphlet

    It seems, and this could be the crucial point
    flaw in our arguement, that disciplines strongly
    diverge in their peer-reviewing quality. This
    would demand a more customized approach
    to redesigning the system.
    Reply With Quote  
     

  57. #56  
    Universal Mind John Galt's Avatar
    Join Date
    Jul 2005
    Posts
    14,169
    Quote Originally Posted by Kevin Clever View Post
    Sorry 'bout that. Here's an interesting reference
    supporting some of my views. While the author,
    by the way far more experienced than me, seems
    to have given up on the subject, I do not like his
    conclusions. We need to think hard to finally fix
    the system, and I believe a joint effort will bring
    drastic change to 21st century publishing.

    Peer review: a flawed process at the heart of science and journals
    Thank you for your response and the link. However I see nothing in the link that addresses my question. The author is discussing peer review. I am questioning one of the premises on which you base the need to modify or replace peer review.

    So here is the question, for the fourth time, "Do you have peer reviewed research that would tend to support your assertion that a large proportion of scientific articles are not worth reading ? "

    Reply With Quote  
     

  58. #57  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by John Galt View Post
    Quote Originally Posted by Kevin Clever View Post
    Sorry 'bout that. Here's an interesting reference
    supporting some of my views. While the author,
    by the way far more experienced than me, seems
    to have given up on the subject, I do not like his
    conclusions. We need to think hard to finally fix
    the system, and I believe a joint effort will bring
    drastic change to 21st century publishing.

    Peer review: a flawed process at the heart of science and journals
    Thank you for your response and the link. However I see nothing in the link that addresses my question. The author is discussing peer review. I am questioning one of the premises on which you base the need to modify or replace peer review.

    So here is the question, for the fourth time, "Do you have peer reviewed research that would tend to support your assertion that a large proportion of scientific articles are not worth reading ? "


    I would like not to denounce particular flaws that I have encountered in
    my colleagues works.

    However, sometimes you get manuscripts that are full of typos, with
    cryptic figure captions and unreadable axis labels. More subtle, even
    scientific mistakes, are usually (~30%) not caught by the reviewer
    as the cited study suggests. The system needs to give the author
    a little bit more time to publish the work, so that people will understand
    when they read.

    Ah, and then there's these authors of which you read one article
    and the next 10 of them are very much alike. Salami publishing
    sometimes happens, but I suppose that some authors do that purpose-
    fully because it raises the chance of reaching a higher-level journal
    with some of the pizza slices.

    The problem that I have with this is that among the dozens of articles
    published every week in your field, you might actually miss something
    important.

    How do you prevent that?
    Reply With Quote  
     

  59. #58  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Also, I'm not saying that my research is bullet proof:
    I'm subject to the same flawed system that drives you
    to publish all the time, and then also to write grants.

    In the end, researchers are busy writing and don't get
    to, ever, see their lab table again. Many of the older
    colleagues are in this category, and I don't think that
    this is overly efficient, i.e. I don't see the benefit.
    Reply With Quote  
     

  60. #59  
    Universal Mind John Galt's Avatar
    Join Date
    Jul 2005
    Posts
    14,169
    Quote Originally Posted by Kevin Clever View Post
    I would like not to denounce particular flaws that I have encountered in
    my colleagues works.
    Thank you for your prompt response.

    I can quite understand that you would not wish to point out specific deficiencies you have encountered. That would raise, perhaps, ethical issues and would certainly be uncomfortable. However, I did not ask you to do that. I specifically asked if you have peer reviewed research that would tend to support your assertion that a large proportion of scientific articles are not worth reading?


    Please tell me in what way my request, which seems a simple one to me, is not clear.

    I understand that
    you think a large proportion of scientific articles are not worth reading. I am not interested in your opinion as a basis for changing the peer review system. For me to consider this I need more than anecdotal evidence to support such a move. (I am rather surprised that as a scientist you do not seem to appreciate that.)

    So, for the fifth time, do you have peer reviewed research that would tend to support your assertion that a large proportion of scientific articles are not worth reading?

    Please Kevin, don't make me ask for this a sixth time. If you don't have this research, that's fine. I'm not convinced by that premise - it is a flawed premise. But if you do have it, may we be told about it?
    Reply With Quote  
     

  61. #60  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by John Galt View Post
    Quote Originally Posted by Kevin Clever View Post
    I would like not to denounce particular flaws that I have encountered in
    my colleagues works.
    Thank you for your prompt response.

    I can quite understand that you would not wish to point out specific deficiencies you have encountered. That would raise, perhaps, ethical issues and would certainly be uncomfortable. However, I did not ask you to do that. I specifically asked if you have peer reviewed research that would tend to support your assertion that a large proportion of scientific articles are not worth reading?


    Please tell me in what way my request, which seems a simple one to me, is not clear.

    I understand that
    you think a large proportion of scientific articles are not worth reading. I am not interested in your opinion as a basis for changing the peer review system. For me to consider this I need more than anecdotal evidence to support such a move. (I am rather surprised that as a scientist you do not seem to appreciate that.)

    So, for the fifth time, do you have peer reviewed research that would tend to support your assertion that a large proportion of scientific articles are not worth reading?

    Please Kevin, don't make me ask for this a sixth time. If you don't have this research, that's fine. I'm not convinced by that premise - it is a flawed premise. But if you do have it, may we be told about it?

    I would assume that I have read about 500 articles in my field.
    That's a neglegible fraction, but still I get away with it pretty well.
    Does that prove the point (maybe only that I'm a bad scientist)?
    Reply With Quote  
     

  62. #61  
    Universal Mind John Galt's Avatar
    Join Date
    Jul 2005
    Posts
    14,169
    Kevin, do you have learning difficulties? Excuse my abruptness, but this is frigging ridiculous. I am asking you to produce citations for research where the researchers have established, by some means, that a large proportion of scientific articles are not worth reading. I am asking for independent confirmation by a team of researchers that your personal opinon that a large proportion of scientific articles are not worth reading is substantiated by that research.

    What do I have to say to make that clear? I know you have read many papers. I know you think a large of proportion of them are not worth reading. I want to know if others have reached the same conclusion and have demonstrated this by a properly constructed research project. I want you to cite such research. I did not ask you if you had in your possession peer reviewed research you had read in order to form the opinion. That is a given. I asked about the frigging properly conducted peer reviewed research carried out by others to substantiate your view.

    Given your inability to understand this simple request on the sixth time of asking I am left wondering if that contributes to the fact that you find many papers not worth reading.

    Now, can I take it from your last post that you do not have and are unaware of any research of the type I have been asking you for? In which case you are asking me to support a change in process based upon your unsubstantiated opinion. Do you think that is scientific?
    Reply With Quote  
     

  63. #62  
    Forum Freshman
    Join Date
    Feb 2013
    Posts
    29
    Quote Originally Posted by John Galt View Post
    Kevin, do you have learning difficulties? Excuse my abruptness, but this is frigging ridiculous. I am asking you to produce citations for research where the researchers have established, by some means, that a large proportion of scientific articles are not worth reading. I am asking for independent confirmation by a team of researchers that your personal opinon that a large proportion of scientific articles are not worth reading is substantiated by that research.

    What do I have to say to make that clear? I know you have read many papers. I know you think a large of proportion of them are not worth reading. I want to know if others have reached the same conclusion and have demonstrated this by a properly constructed research project. I want you to cite such research. I did not ask you if you had in your possession peer reviewed research you had read in order to form the opinion. That is a given. I asked about the frigging properly conducted peer reviewed research carried out by others to substantiate your view.

    Given your inability to understand this simple request on the sixth time of asking I am left wondering if that contributes to the fact that you find many papers not worth reading.

    Now, can I take it from your last post that you do not have and are unaware of any research of the type I have been asking you for? In which case you are asking me to support a change in process based upon your unsubstantiated opinion. Do you think that is scientific?

    See you smwhr else.
    Reply With Quote  
     

  64. #63  
    Forum Radioactive Isotope Paleoichneum's Avatar
    Join Date
    Oct 2008
    Location
    Washington State, USA
    Posts
    4,625
    Here is the Facebook link for Kevins group http://www.facebook.com/groups/543896462309094/ I see he has not actually posted a link to this discussion, or changed his stance based on the suggestions critiques raised here.
    If more of us valued food and cheer and song above hoarded gold, it would be a merrier world. -Thorin Oakenshield

    The needs of the many outweigh the need of the few - Spock of Vulcan & Sentinel Prime of Cybertron ---proof that "the needs" are in the eye of the beholder.
    Reply With Quote  
     

  65. #64  
    Universal Mind John Galt's Avatar
    Join Date
    Jul 2005
    Posts
    14,169
    Quote Originally Posted by Kevin Clever View Post
    See you smwhr else.
    A good scientist may find criticism very uncomfortable, but they understand that it is essential to their ability to perform high quality and relevant research. I am disappointed that you have chosen to run away rather than take that criticism head on. I will let you reflect on the significance of that decision.
    Reply With Quote  
     

  66. #65  
    Forum Professor river_rat's Avatar
    Join Date
    Jun 2006
    Location
    South Africa
    Posts
    1,497
    My 2 cents: a lot of research is rather error riddled. First study of this type that springs to mind : Emili Garcia-Berthou and Carles Alcaraz, Statistical Errors, BMC Medical Research Methodology 2004, 4:13

    I think it was a Wall Street Journal story a while ago about the large increase in the number of retractions over the last few years. (ah found it - Mistakes in Scientific Studies Surge - WSJ.com)
    As is often the case with technical subjects we are presented with an unfortunate choice: an explanation that is accurate but incomprehensible, or comprehensible but wrong.
    Reply With Quote  
     

  67. #66  
    Universal Mind John Galt's Avatar
    Join Date
    Jul 2005
    Posts
    14,169
    Thank you for the link RiverRat. I have no doubt that the peer review process could be improved. My reservation was that Kevin seemed to be basing the need purely personal experience and subjective opinion. At no time, as far as I saw, did he say the reason most papers were not worth reading was because they were error ridden. They were just not worth reading. For a practicing scientist I found him very unscientific in his approach.

    Even if we consider that the retracted papers represent only a small proportion of those with errors this still seems insufficient to claim that a large proportion of papers are not worth reading. It is also worth considering that this upswing in retractions may reflect a greater readiness to admit errors, in which case things may developing in the opposite direction to what Kevin claimed. These are all only possibilities - which is why relevant research of the type you've linked to is essential, but which Kevin was incapable of providing.
    Reply With Quote  
     

Similar Threads

  1. Will Iran strike first?
    By westwind in forum Politics
    Replies: 11
    Last Post: February 7th, 2012, 12:44 AM
  2. Rejected!
    By Swordsmith in forum General Discussion
    Replies: 9
    Last Post: October 17th, 2007, 07:51 PM
  3. Publications Online - Call for Reviewers
    By PubOnline in forum Links
    Replies: 0
    Last Post: July 12th, 2007, 10:53 AM
  4. Replies: 6
    Last Post: August 29th, 2006, 09:16 PM
Bookmarks
Bookmarks
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •