Repository navigation
clone_from progress is broken in newer version #604
Description
Activity
OK, I've narrowed it down through trial and error.
Working correctly: 2.0.8
Broken: 2.0.9Thanks for narrowing it down!
I had a closer look and found this commit which simplified the code to just-in-time parse lines from stdout and/or stderr.To me the code looks good - the main thread blocks while lines are read from streams as they become available. Apparently this line iteration doesn't work as expected for you.
Can you provide more information about your environment as it may relate to the issue:
- the exact python version
- the OS
Thanks a lot
Python 3.5.2
Ubuntu 16.04Any progress on this? Have you been able to reproduce it? It should be trivial to test in a vagrant box or similar.
This is causing us some fairly bad UX issues in our project at the moment.
Full steps to reproduce:
Install Vagrant.
Save the attached Vagrantfile into a new folder (you'll need to remove the .txt extension thanks to Github's dumb mimetype detection).
Vagrantfile.txtRun
vagrant up && vagrant ssh
Inside the vagrant box:sudo apt update sudo apt install python3-pip -y pip3 install gitpython ssh-keyscan github.com > ~/.ssh/known_hosts python3import git def u(*args): print(args, flush=True) git.Repo.clone_from("https://github.com/gitpython-developers/GitPython.git", "test", u)Stills broken. Works in 2.0.8.
I'm using standard Progress class.class Progress(RemoteProgress): def update(self, op_code, cur_count, max_count=None, message=''): print('Downloading: (==== {} ====)\r'.format(message))and then calling it
Repo.clone_from("git://github.com/{}".format(link.split('.com/')[1]), PATH_NAME, progress=Progress())Python 3.4.4.1Qt5 from WinPython
Windows 8.1 64bitsCould you try it again with the latest version?
kind of working 2.1.7
working
print('Downloading: (==== {} ====)'.format(message)
not working
print('Downloading: (==== {} ====)'.format(message), end='\r')Nope, no change. Followed my exact steps mentioned a couple of comments ago, and got exactly the same problem. GitPython==2.1.7
#604 (comment)Confirmed, this is the same problem, and is fixed by following @mrozekma's comment:
I fixed it locally by changinggit.remote.add_progressto also setkwargs['universal_newlines'] = True, and changinggit.base.Repo._cloneto passdecode_streams=Falsetohandle_process_outputI made a tweak to those instructions to fix build issues in the referenced pull request, though there are still a couple of build errors which mean nothing to me.
Actually, it looks like the remaining build issues also exist in the main branch. So, please review and merge.
@Dreamsorcerer I am closing this issue as its fix will be contained in the upcoming release.
Still broken in 2.1.8
Reverting to last known working version 2.0.8Are you following my exact instructions in my previous comment #604 (comment)? I've just tested it again following those exact instructions, and it it is working correctly. If you are encountering an issue in a different situation, then it is a different bug, please file a new one.
To use a workaround for another bug, I had to first update GitPython. After updating with
pip3 install --upgrade gitpython, which installed 2.1.3, the progress feature no longer functions correctly.With this code:
While running it on a fairly large repository, I get the first few updates, upto:
(10, 16118.0, 16118.0, '')(which means compressing has completed)But, then the next stage when it is downloading everything, which can take about 15 mins, there is no feedback until it has completed that stage, then all the missing feedback is sent at once, as if the feedback has gathered in a buffer and not sent until the current stage is complete.