-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add apk url and version code for latest release #6
Conversation
This provides the data for TeamNewPipe/NewPipe#1520
api.py
Outdated
} | ||
|
||
repo = json.loads(repo_data) | ||
|
||
# scrap latest GitHub version and apk from website | ||
elem = html.fromstring(release_github_data) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@TheAssassin Do you have any idea how I can reduce the redundancy here? I didn't find a good way to do so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What kind of redundancy are you talking about specifically?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have the same structure for getting the GitHub version, apk and the contributors count:
Get HTML, use cssselect
to filter the HTML, check if the we get the a valid number of results, get the first result and modify it.
Unfortunately I couldn't move this to a new method because I failed to execute the steps in the try
blocks in a good way. My initial idea was to return tags[0]
:
def scrape_data_from_html(data, selector: str):
elem = html.fromstring(data)
tags = elem.cssselect(selector)
if len(tags) < 1:
return -1
else:
try:
return tags[0]
except:
return -1
But than I realised that I'd needed another try and catch to handle the possible exceptions which can be thrown by tags[0].get('href')
, tags[0].text
and int(tags[0].text)
:
contributors_result = scrape_data_from_html(contributors_data, ".numbers-summary a[href$=contributors] .num")
try:
contributors_count = int(contributors_result.text)
except:
contributors_count = -1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am okay with this as it is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good,
@@ -166,7 +192,13 @@ def assemble_release_data(data: str): | |||
"translations": int(translations["count"]), | |||
}, | |||
"flavors": { | |||
"stable": assemble_release_data(stable_data), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it make sense to split this up into "github": {"stable": {...}, ...}
and "f-droid": {"stable": {...}, ...}
instead? GitHub names releases differently, i.e. there's releases and pre-releases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good.
api.py
Outdated
except: | ||
release_github_version = -1 | ||
|
||
# scrap contributors from website |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's "scrape" ("extract"), not "scrap" ("dispose", "throw away").
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can be merged, @TobiGr.
fix typos
@TheAssassin Feel free to merge. I can't do it. |
@TobiGr I was already working on some solution for this. I added a team, and I'm granting this team permissions for all relevant NewPipe repositories. Please check if you can merge here now, too. |
That is a good idea! |
Deployment in progress. |
Thanks |
This should be everything we need for TeamNewPipe/NewPipe#1520
I changed the structure a bit to handle the different release platforms F-Droid and GitHub better.