Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why rubinius is always cloned and failed? #19

Open
zhangysh1995 opened this issue Mar 14, 2018 · 1 comment
Open

Why rubinius is always cloned and failed? #19

zhangysh1995 opened this issue Mar 14, 2018 · 1 comment

Comments

@zhangysh1995
Copy link

The errors:

zhangysh1995@ubuntu-zhangyushao:~$ crawld -c crawld.conf 
E0314 10:19:15.173304    6197 crawld.go:79] impossible to clone https://github.com/rubinius/rubinius.git in /var/crawld/c/rubinius/rubinius (network error) skipping
E0314 10:19:16.711802    6197 github.go:253] GET https://api.github.com/search/repositories?per_page=100&q=language%3Ac&sort=stars: 401 Bad credentials []
E0314 10:19:17.156407    6197 github.go:253] GET https://api.github.com/search/repositories?per_page=100&q=language%3Acpp&sort=stars: 401 Bad credentials []

After these two lines of information, the program is running but cannot see any outputs.

My configuration file:

{
    "database": {
        "hostname": "localhost",
        "port": 5432,
        "username": "crawler",
        "password": "crawler",
        "dbname": "crawler",
        "ssl_mode": "disable"
    },
    "clone_dir": "/var/crawld",
    "crawling_time_interval": "1h",
    "fetch_time_interval": "5m",
    "fetch_languages": [
        "c",
        "cpp"
    ],
    "tar_repositories": false,
    "tmp_dir": "/ramdisk",
    "tmp_dir_file_size_limit": 2.0,
    "max_fetcher_workers": 2,
    "throttler_wait_time": 900,
    "throttler_sliding_window_size": 60,
    "throttler_leak_interval": 1000,
    "crawlers": [
        {
            "type": "github",
            "languages": [
                "c",
                "cpp"
            ],
            "limit": 0,
            "since_id": 0,
            "fork": false,
            "oauth_access_token": "token here",
            "use_search_api": true 
        }
    ]
}
@zhangysh1995 zhangysh1995 changed the title Why rubinius is always clones and failed? Why rubinius is always cloned and failed? Mar 14, 2018
@rolinh
Copy link
Member

rolinh commented Mar 17, 2018

Hi @zhangysh1995,

Not sure. I haven't used this project for about 2 years. I'll try to give it a shot one of these days and figure out the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants