Journal article organization

[ttkUWMadison]

Hi All,

I am a PhD student and had a question. Does anyone have any suggestions for a successful database and journal article storage/organization method? I have well over a hundred journal articles and am wondering how best to store, organize and retrieve them? I currently use Endnote but am not sure that is the best database system for organization.

Thanks for the help.

[gp4100]

Hi All,

I am a PhD student and had a question. Does anyone have any suggestions for a successful database and journal article storage/organization method? I have well over a hundred journal articles and am wondering how best to store, organize and retrieve them? I currently use Endnote but am not sure that is the best database system for organization.

Thanks for the help.

In case you have a Mac, I like Bookends a lot (http://www.sonnysoftware.com).

Cheers,

Michael

[American in Beijing]

I'm currently trying out Mendeley and I like it a lot, though I feel that its resources for the sciences are unfortunately a lot greater than those for the humanities, . I've never used Endnote, so I can't really compare.

[socialpsychg]

I'm a fan of Zotero (a firefox addon) because it was made with the internet in mind (and it's free!).

[repatriate]

I use Papers, which is for Mac only. It has great sorting and search features and integrates with several search engines. Plus, if you get the iPhone/iPad app you can sync your library and have your PDFs with you.

[bluellie]

As far as database storage goes, I've been using a copy of DevonThink to store and organize my recipes. It's got a pretty decent search functionality and will suggest where you might want to store a new article. I'm not sure if I'd pay money for it (granted, I'm using an older version I got it for free during one of those MacHeist promotions), but it's a solid piece of software.

[evolve]

I too use Papers and love it. I experimented with Mendeley for a bit, but ultimately chose to stick with Papers. Of course, as already stated, Papers is a Mac-only program though.

[Eigen]

For a PC, I have found Endnote to be great. I use it to keep all my references organized (being able to attach up to 45 different files to a reference is great), and just used it for an 87 reference review paper I wrote. It worked flawlessly.

It's not free, but then it's really cheap, and most departments/schools probably have it for a discount or free.

The other nice thing about Endnote, is that the mac and PC versions work well together, which means my boss (who uses a mac) and I can easily share references/reference libraries.

[Riotbeard]

I'm a fan of Zotero (a firefox addon) because it was made with the internet in mind (and it's free!).

My grad program told me to get zotero, haven't yet but will.

[timuralp]

If you guys want to organize a folder/look at how to deal with pdfs in a programmable way, here's a cool post for dealing with this in python: http://en.dogeno.us/2010/02/release-a-python-script-for-organizing-scientific-papers-pyrenamepdf-py/

The idea is to try to find the DOI and then look-up the information about the document. I'm trying to write something that would try to grab the title as the first line. At least that's the way a lot of CS articles are formatted.

[ttkUWMadison]

Hey, thanks everyone for replying and the great suggestions! I have downloaded and am going to try Mendely (non Mac user here).

Appreciate your help!

[Branwen daughter of Llyr]

Hey, thanks everyone for replying and the great suggestions! I have downloaded and am going to try Mendely (non Mac user here).

Appreciate your help!

Mendely is pretty awesome - I just started using it this week, and have been extremely pleased with it so far. I have yet to test it's citation plug-in with Word (that will happen next month), but I'm sure it will be helpful in organizing the copious amounts of PDFs I'm saving for my next research paper (and I'm not even in grad school yet...)

[natsteel]

I use Papers, which is for Mac only. It has great sorting and search features and integrates with several search engines. Plus, if you get the iPhone/iPad app you can sync your library and have your PDFs with you.

I'm in History and have well over 600 pdfs (mostly journal articles) and I use Papers for Mac. The search features are fantastic. On my PC desktop, I downloaded Mendeley but haven't had the time to enter info for so many PDFs, so I'll just keep using EndNote, I guess.

[Branwen daughter of Llyr]

I'm in History and have well over 600 pdfs (mostly journal articles) and I use Papers for Mac. The search features are fantastic. On my PC desktop, I downloaded Mendeley but haven't had the time to enter info for so many PDFs, so I'll just keep using EndNote, I guess.

Mendeley interfaces with google scholar for PDF info - it found the correct information for all but one of the 20 PDFs I saved in collections this week. I only had to manually input info for that single article (which was from 1912, LOL). Every time you add a PDF to a collection in Mendeley, it retrieves info from the file itself and asks you to confirm the info, or search with google scholar according to the title. Pretty nifty.

[fuzzylogician]

Mendeley interfaces with google scholar for PDF info - it found the correct information for all but one of the 20 PDFs I saved in collections this week. I only had to manually input info for that single article (which was from 1912, LOL). Every time you add a PDF to a collection in Mendeley, it retrieves info from the file itself and asks you to confirm the info, or search with google scholar according to the title. Pretty nifty.

Maybe I'm doing something wrong because I tried uploading ~20 pdfs using Mendeley and in pretty much all of the cases the program extracted incorrect data (or none at all) and I had to input the information manually. I've tried importing files both from my computer and from google scholar. Mendeley also doesn't include the citation style most common in linguistics and I couldn't find how to define it as a new citation style. I really fail to see the upside to using this program. Also, I'm reluctant to use Endnote because I would have to buy the program first and only then will I be able to tell if the citation style I need is available and if it extracts the info from my pdfs automatically.

Edited June 24, 2010 by fuzzylogician

[Eigen]

Endnote will not extract the information from your PDFs automatically.

I have yet to find a journal that did not have a function to export a citation to endnote, however.

It took me about 3 hours one day to migrate my 200ish PDFs into endnote- I just had to look up each journal again, and download a citation.

Once you're already using it, you download a PDF and the citation at the same time.

As for the citation formats- I'd bet that it has the one you're looking for. It has every journal format I've ever even heard of in Chemistry, and you can easily create new formats or modify the existing ones. Its literally thousands of citation formats pre-loaded, however.

I also find it quite common for a journal to have an endnote template available that they request you use for submissions.

If your library connection allows it, endnote will also browse and find the full text for any citations you've entered.

I'd be quite surprised if your department/school didn't have a license available to it, however. Have you checked with your department or the library?

Edited June 24, 2010 by Eigen

[fuzzylogician]

<snip>

I'd be quite surprised if your department/school didn't have a license available to it, however. Have you checked with your department or the library?

Thank you for the reply. So I guess EndNote is always a valid option. Unfortunately:

EndNote at MIT

MIT does not offer a site license or discount pricing for EndNote. The MIT Libraries offer limited support for EndNote users, particularly with issues involving EndNote connectivity with library-supported resources. The libraries also provide introductory hands-on training in the use of EndNote during IAP and by arrangement.

Members of the MIT community do have free access to EndNoteWeb, a web-based version of EndNote. However, we recommend trying RefWorks instead of EndNoteWeb if you are planning to use a web-based product.

Does anyone here have any experience using RefWorks and can recommend it?

Do you find it problematic to only have an online resource?

Edited June 25, 2010 by fuzzylogician

[timuralp]

Maybe I'm doing something wrong because I tried uploading ~20 pdfs using Mendeley and in pretty much all of the cases the program extracted incorrect data (or none at all) and I had to input the information manually. I've tried importing files both from my computer and from google scholar. Mendeley also doesn't include the citation style most common in linguistics and I couldn't find how to define it as a new citation style. I really fail to see the upside to using this program. Also, I'm reluctant to use Endnote because I would have to buy the program first and only then will I be able to tell if the citation style I need is available and if it extracts the info from my pdfs automatically.

The problem of extracting the right information out of PDFs is actually pretty hard because a lot of PDFs have nonsense set in metadata and parsing the thing is a pain. One way to do it is by looking up a DOI and finding the right citation (see the link I posted before and a number of articles have no DOI specified), but the only other way I could figure out was to write or use something to search for the text online and try to grab a citation there. I'd be curious if anyone knows of a piece of software that can actually that data out well. So far I've been tinkering with pyPdf and it looks like I might be able to pull out the names of some papers and authors but on some files it doesn't parse correctly

[Eigen]

And it gets even more difficult if the names/titles have unusual symbols.

The other thing to do is watch the Endnote website. They have sales pretty regularly for students- and honestly, even the normal price isn't that bad.

You can also download a free trial from the Endnote website, if you want to try it out.

I have Endnote X2, and I've been thinking about upgrading to X4 on my own- I think Endnote Student is somewhere around $100, which for as often and as heavily as I use it (every day, often constantly open on my computer), it's well worth it.

I didn't like Endnote Web as much, or Refworks- I prefer to have a hard copy of my database saved to my flash drive as a backup in case something happens to it. And while rare, I've heard of the online reference databases experiencing errors and data wipes- and even a small risk isn't worth it to me, not when it comes down to losing my whole reference library.

Thankfully, Endnote is exceptionally easy to back up. It's one file for all of your references and their associated documents.

is the link to student pricing from Endnote. It's $115 from Thompson Reuters, if MIT doesn't offer affiliations with any of the other student sites.

I'd also check here to see if endnote is offered at a discount through GovConnection.

Edited June 25, 2010 by Eigen

[Eigen]

Actually, I was looking at the X4 edition of Endnote today...

And it says that it can import a file of PDFs and extract the bibliographic information from them....

Import and Search PDFs: Import a file or a folder of PDFs directly into your EndNote library. The basic bibliographic data is extracted for most PDFs and you can even search the contents of a PDF.

A few other useful features of the newer versions of Endnote:

You can attach individual figures to the citation, coupled with their captions, and then directly copy them to a new document.

And also, you can "mass edit" your references, and add a new tag to every reference in a subgroup or library, either appending it to an existing field, or adding a new field. Quite useful for those times when you decide to re-organize your structure.

[profound_g]

Let this be another vote for Zotero. For me at least, it does everything I need automatically. Its only flaw is that it doesn't take an outline an write an article on its own. I'm looking at the mendeley site and it seems quite similar; didn't do an actual comparison though.

Edited June 27, 2010 by profound_g

[Dr. Gunn]

The problem of extracting the right information out of PDFs is actually pretty hard because a lot of PDFs have nonsense set in metadata and parsing the thing is a pain. One way to do it is by looking up a DOI and finding the right citation (see the link I posted before and a number of articles have no DOI specified), but the only other way I could figure out was to write or use something to search for the text online and try to grab a citation there. I'd be curious if anyone knows of a piece of software that can actually that data out well. So far I've been tinkering with pyPdf and it looks like I might be able to pull out the names of some papers and authors but on some files it doesn't parse correctly

Hi All! I hope you don't mind me barging in on this thread. I'm the academic community liaison for Mendeley, so I can answer some of the questions people are asking.

The idea behind Mendeley is to get you up and running as soon as possible with a minimum of data entry, so you can just point it at the directory that has your PDFs and it will extract the citation information for you, optionally renaming and organizing the files, too. As timuralp noted, getting the metadata right is challenging, but we have several different ways of getting the right info - Extraction from the XMP metadata, Google Scholar search on the title, or lookup of the DOI, PMID, or ArxivID from Crossref, Pubmed, or Arxiv.org. If all of these fail, you can manually enter the missing info. Like Branwyn says, it generally works pretty well. You can also fetch information directly from Zotero or from exported Endnote files.

In terms of the comparison to Zotero, Papers, and Endnote (of course I'm biased) but the major differences are:

• Mendeley vs. Zotero - Mendeley is a standalone program, so you don't have to use Firefox for it to work and your data isn't subject to browser crashes.
  • Mendeley vs. Endnote - Mendeley is free and designed using modern design principles. Little manual data entry is required and you can search the full text across all your papers. Mendeley also has annotation, sharing & discovery tools lacking in Endnote.

  [*]Mendeley vs. Papers - Mendeley is free and cross-platform, so you can collaborate with anyone, regardless of OS. Also, while Papers organizes files very nicely, Papers doesn't have an add in for Word and OpenOffice.

  Here's a less biased version comparing all the major reference managers: http://blogs.nature....ence-management

  Hope this was helpful! Please don't hesitate to get in touch (william.gunn@mendeley.com) if you have any followup questions.

Edited June 29, 2010 by Dr. Gunn

[Eigen]

Thanks for the comparison, it was quite helpful.

To those reading the link, do note that it seems to be on Endnote X3, not the new X4 version, as X4 can extract information and create citations from a PDF.

[Dr. Gunn]

Thanks for the comparison, it was quite helpful.

To those reading the link, do note that it seems to be on Endnote X3, not the new X4 version, as X4 can extract information and create citations from a PDF.

Hey, Eigen, I got my PhD at Tulane. How are you liking it?

[Eigen]

Quite a bit- it's a nice school, and the department is great.

I find it interesting coming from a large state school; here the graduate population is almost as large as the undergraduate one, so there is a lot more attention paid to graduate students in the overall school design, I think.

You should convince the Tulane library to offer Mendeley support/classes along with Refworks and Endnote- I think it would get a lot larger portion of the school using it.