Just looking at your code briefly in-between college football Saturday
but it looks like you are closing your file inside your loop where you
are adding easy handles to the multi. I think that's probably the
source of your crash since the FILE* is invalid when the getpage call
is done. Make an array of FILE*'s. Keep them around until you're done.
Have each of them point to a different file (which looks like its also
an issue in the code). Good luck!

On Oct 3, 2009, at 12:10 PM, will griffiths wrote:

> Hello,
>
> I have written a full text indexing program in C++ and am now
> building a spidering program and I intend to use libcurl for the
> download side of things.
>
> I am struggling a bit to transfer my curl multi code from PHP to C++
> as I relied on the "getcontent" function quite heavily.
>
> I have included the full source code below. When i turned verbose on
> I got this output:
> * Closing connection #0
> * Closing connection #1
> * Closing connection #2
> followed by a crash. I probably need help understanding how the curl
> functions work in C++. I have read the docs for the functions
> "individually" and then strung them together by looking at egs in
> Google Code Search.
>
> Any help appreciated :)
> Will
>
> //#pragma comment(lib, "libcurl.lib") // now appending /link <DIR>
> \libcurl.lib to cmd args instead
>
> #include <string>
> #include <iostream>
> #include <map>
> #include "curl/curl.h"
>
> using namespace std;
>
> FILE *fd;
>
> static size_t getpage(void *incoming, size_t size, size_t nmemb,
> void *page)
> {
> return fwrite(incoming, size, nmemb, (FILE *)page);
> }
>
> int main()
> {
>
> map<int, string> pass;
> pass[0] = "http://chemrefer.com/index.html";
> pass[1] = "http://www.chemrefer.com/chemistry_search.php?page_title=news&submit
> ";
> pass[2] = "http://www.chemrefer.com/chemistry_search.php?page_title=toolbar&submit
> ";
> pass[3] = "http://www.chemrefer.com/chemistry_search.php?page_title=about&submit
> ";
> pass[4] = "http://www.chemrefer.com/chemistry_search.php?page_title=contact&submit
> ";
>
> CURL *h[5];
>
> CURLM *mh;
> mh = curl_multi_init();
>
> for (int i = 0; i < 5; ++i) {
>
> char *urlbuf = new char[pass[i].length()];
> strcpy(urlbuf, pass[i].c_str());
>
> h[i] = curl_easy_init();
> fd = fopen("download.txt", "ab");
>
> curl_easy_setopt(h[i], CURLOPT_URL, urlbuf);
> curl_easy_setopt(h[i], CURLOPT_WRITEFUNCTION, getpage);
> curl_easy_setopt(h[i], CURLOPT_WRITEDATA, fd);
> curl_easy_setopt(h[i], CURLOPT_HEADER, 0);
>
> curl_multi_add_handle(mh, h[i]);
>
> delete[] urlbuf;
>
> fclose(fd);
>
> }
> pass.clear();
>
> int running;
> CURLMcode result;
> do {
> result = curl_multi_perform(mh, &running);
> if (result != CURLM_CALL_MULTI_PERFORM) break;
> }
> while (running > 0);
>
> for (int i = 0; i < 5; ++i) curl_multi_remove_handle(mh, h[i]);
> for (int i = 0; i < 5; ++i) curl_easy_cleanup(h[i]);
> curl_multi_cleanup(mh);
>
> }
>
>
> -------------------------------------------------------------------
> List admin: http://cool.haxx.se/list/listinfo/curl-library
> Etiquette: http://curl.haxx.se/mail/etiquette.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
Received on 2009-10-03