1

There are more than 1000 files in a storage server account and when I use curl to display the list of files, output is limited to only first 1000.

Just to test, I had deleted a file showing up in the GET list and then did a GET again, this time curl did display 1000 files with an additional file from full list.

Is there any way to tell curl to display full list instead of limiting to 1000?

Edit:

curl -X GET -u <user>:<passwd> <url-full-path-of-container>

curl version: curl 7.29.0 (x86_64-redhat-linux-gnu)

I am not sure whether problem is on server side because I dont really have full privileges to check that.. Can it be on server side too?

3
  • Can you show the exact command you are using? I used curl v7.19.7 and did not get this behaviour. Possibly a limit of the web server rather than curl? That's pure specuation of course.
    – Unbeliever
    Oct 13 '16 at 6:46
  • Are you sure the problem is with curl, and not on the server?
    – Zoredache
    Oct 13 '16 at 6:54
  • 2
    I think this has to be on the server side. There is no reason I can think of for curl to do this.
    – Unbeliever
    Oct 13 '16 at 6:57
1

You should mention the protocol you're been using with curl. The -X command restrict the list to: HTTP, FTP, POP3, IMAP. Which one? Why are you using -X?

Anyway, it's possible that the remote server is limiting the list: curl has no intrinsic internal limit on listing AFAIK.

I'd suggest you to try an interactive session (especially if it's FTP) by using another command line tool or by elevating the debugging level with --verbose option.

1

It's a common and recommended practice for web services to paginate their responses. That is, to limit the number of results returned in a response, and let clients pass a parameter to get the next page. Some services may allow changing the limit (= page size), up to some hardcoded upper limit.

curl by itself doesn't impose limits on responses. The limit that you have found is certainly at the server side. Whether the server supports the additional parameters to customize the page size or to specify the page (other than the first), depends entire upon the implementation of the service. I would look for that information either in the API documentation, or else in the response headers.

This site is temporarily in read only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .