[fpc-other] GIT versioning server on Raspberry Pi?

[Bo Berglund]

I have followed parts of the very long thread "GIT & SVN" and found it
informative and partly amusing too...
Since I am still on CVS it really was getting me to think about new
version control systems and especially GIT since Graeme is pushing so
hard for it.

Background:
-----------
I have a small company where the engineering team is myself (not so
much anymore) and two engineers doing software and electronics. We are
making embedded systems and PC (Windows) based processing software.

Back in 2002 in order to safeguard against various catastrophies I put
in place a CVS *server* on our network for managing the versions of
our code and also our CAD drawings and documentation etc. So a lot of
the files in CVS are binary. The instructions and firmware image files
for our production people are also on our CVS.
The CVS backend is an old CVSNT release on a Windows Server 2003 and
we use the same cvsnt as client but via the WinCVS GUI front-end.

Problems to solve:
------------------
- We need to get a system that is supported today and preferably on a
platform that will not be obsoleted like the Windows versions are.

- Since we are also now doing development in Linux (for Raspberry Pi
using FPC) we need something running on Linux (for the clients).

- Generally I don't want us to be ancient tool-wise, but at the same
time we have such a huge amount of information invested into the CVS
repository. So I need the end soultion to hve some functionality like
the CVS system we use today. And a possibility to import the CVS
stuff.

Testing
--------
So I thought that one way to go forward was for me to use an RPi3 box
as a repository server with GIT or whatever (seems like the CVS
successor SVN is now also on the decline...).
Is there a good way to set up an RPi3 box as a GIT server and get
going with that?

Can GIT work in a way that would be comparable to CVS regarding
concurrent development etc?

Any suggestions on where to start?

[Bo Berglund]

On Mon, 29 May 2017 12:01:32 +0000, Mark Morgan Lloyd

Testing--------So I thought that one way to go forward was for me to use an RPi3 boxas a repository server with GIT or whatever (seems like the CVSsuccessor SVN is now also on the decline...).Is there a good way to set up an RPi3 box as a GIT server and getgoing with that?
Can GIT work in a way that would be comparable to CVS regardingconcurrent development etc?
Any suggestions on where to start?

Frankly, and after having run an RPi3 as my desktop system for roughly
nine months, I'd not use an RPi for that. I'd use something like an HP
Microserver with at least mirrored discs, and I'd make sure that I was
able to write a consistent snapshot onto something like a USB stick
storage device.
My recollection is that while you can set up (software-controlled)
mirrored devices on an RPi, you run into problems mounting them at boot
time due to deficiencies in the loader- that was certainly the case when
I looked at it in the latter part of last year, although low-level
firmware improvements (roughly last November) might have fixed some of that.
In addition, it's becoming increasingly obvious to many people that the
power arrangement is manky, and that you risk permanent damage to an
SDCard if inserting a USB device briefly pushes the voltage out of spec-
see discussion of the vcgencmd command.
Finally, you /will/ need to backup, and if you've accumulated any amount
of data you really don't want to try doing that through a single
(internal) USB hub with no possibility of expansion.

OK, I did not have in mind to use an RPi3 as the final system. I just
wanted to acquaint myself with GIT using a small portable unit over
the summer.
If it did work and I found it useful I would put a Linux server on our
VMWare box and use that as the new versioning server. But over the
summer I am moving to a place where space is limited and I thought a
small box like a Pi3 could work as a test bed.

[Graeme Geldenhuys]

Post by Bo Berglund
OK, I did not have in mind to use an RPi3 as the final system. I just
wanted to acquaint myself with GIT using a small portable unit over
the summer.

In that case, you don't even need a RPi... Simply practice by doing the
following on your local system:

mkdir /data/myremote.git
cd /data/myremote.git
git init --share --bare .

You just created your remote [bare] git repository. The term "bare"
means that there is no checked out files. If you view that directory it
simply shows all the normally hidden files that git uses to manage a
repository.

Now for your working repository where you will do you day-to-day commits:

cd /data/devel/
git clone /data/myremote.git/ mywork

You will now have a Git repository in /data/devel/mywork/ which is a
clone of the /data/myremote.git repository. It as automatically set up
the "origin" as your Remote Git Repo. So if you to a git-push, the data
will go to "origin"

Now playing around in this "mywork" repository. Create commits and
branches. Then do a git-push and magically you will have updated the
"origin" remote git repo too (which in this case is /data/myremote.git/)

What you learn here, is exactly how you would do things with a real
remote repository.

I highly recommend everybody interested in using Git reads the free and
open source "Pro Git" book [https://git-scm.com/book/en/v2]. It is
available online or offline in various formats (PDF, ePub etc). Is is
well written, fast to read and with tons of real-world Git examples.

Regards,
Graeme

[Bo Berglund]

On Tue, 30 May 2017 15:37:30 +0100, Graeme Geldenhuys

Post by Graeme Geldenhuys

Post by Bo Berglund
OK, I did not have in mind to use an RPi3 as the final system. I just
wanted to acquaint myself with GIT using a small portable unit over
the summer.

In that case, you don't even need a RPi... Simply practice by doing the

Since my local system is a Windows 7 laptop I have to resort to an RPi
to get the Linux system for which the commands are native...

Post by Graeme Geldenhuys
mkdir /data/myremote.git
cd /data/myremote.git
git init --share --bare .

So this should not be created inside some user home then? In some
documents I found a new user "gituser" is created and in his home is
where they placed the master repository.

Post by Graeme Geldenhuys
You just created your remote [bare] git repository. The term "bare"
means that there is no checked out files. If you view that directory it
simply shows all the normally hidden files that git uses to manage a
repository.
cd /data/devel/
git clone /data/myremote.git/ mywork
You will now have a Git repository in /data/devel/mywork/ which is a
clone of the /data/myremote.git repository. It as automatically set up
the "origin" as your Remote Git Repo. So if you to a git-push, the data
will go to "origin"
Now playing around in this "mywork" repository. Create commits and
branches. Then do a git-push and magically you will have updated the
"origin" remote git repo too (which in this case is /data/myremote.git/)
What you learn here, is exactly how you would do things with a real
remote repository.
I highly recommend everybody interested in using Git reads the free and
open source "Pro Git" book [https://git-scm.com/book/en/v2]. It is
available online or offline in various formats (PDF, ePub etc). Is is
well written, fast to read and with tons of real-world Git examples.

I have read a few chapters on-line, including installing git using the
command:
sudo apt-get install git-all

which (of course) differs from the commands I have found in various
other how-to pages concerning git...

I would very much like to have a PDF copy since I usually find that
easier to read than using on-line webpage versions of books.
Could not find the PDF though...

[Graeme Geldenhuys]

Post by Bo Berglund
Since my local system is a Windows 7 laptop I have to resort to an RPi
to get the Linux system for which the commands are native...

Git is native on all supported platforms now.

Post by Bo Berglund

Post by Graeme Geldenhuys
mkdir /data/myremote.git
cd /data/myremote.git
git init --share --bare .

When you install Git under Windows using the official installer, it
includes Bash Shell integration and a shortcut on the desktop. So the
above commands will work as-is on Windows too.

But you can obviously run a non-Bash shell/console under Windows. Then
simply replace the Unix-style paths with Windows variants. The git-init
command stays the same.

Post by Bo Berglund
So this should not be created inside some user home then?

You can create Git repositories wherever you want and have read/write
access. My "/data" path is simply by 8TB ZFS data pool, where I do all
my work and store all vital data. You can use your $HOME directory
(whatever that translates to on your OS) just as well.

If you are going to share your Git repositories stored on your system
with the public, then better make sure you set up your system's file
permissions correctly, or use a dedicated "shared location" on your
system. Alternatively don't make your laptop or desktop directly
accessible to the public on the Internet, instead push your repository
to a public and secure server somewhere (eg: Github, SourceForge etc).

You mentioned you wanted to play around with Git and get to grips with
it - hence I suggested a local setup without the need of a RPi or some
other device.

Post by Bo Berglund
I have read a few chapters on-line, including installing git using the
sudo apt-get install git-all
which (of course) differs from the commands I have found in various
other how-to pages concerning git...

The Linux distros are to blame for that - more specifically ther
incompatible "package management requirements". I always install git
from source code and compile it myself (like I do with FPC and Lazarus
too). Everything is then included - as it should be. Linux distros f*ck
everything up and split it into multiple packages. eg: git-core,
git-base, git-gui, git-subversion, git-docs etc. G*d damn ridiculous if
you ask me!

Post by Bo Berglund
I would very much like to have a PDF copy since I usually find that
easier to read than using on-line webpage versions of books.
Could not find the PDF though...

I just had a look. The links used to be on the Table Of Contents page,
but for some odd reason they aren't there any more. No stress, The
Internet Archive always comes to the rescue.

This is how it used to look like around 01 March 2017.

http://web.archive.org/web/20170301183218/https://git-scm.com/book/en/v2


The individual eBook downloads from that page still works though.

PDF:
https://progit2.s3.amazonaws.com/en/2016-03-22-f3531/progit-en.1084.pdf

ePub:
https://progit2.s3.amazonaws.com/en/2016-03-22-f3531/progit-en.1084.epub

Mobi:
https://progit2.s3.amazonaws.com/en/2016-03-22-f3531/progit-en.1084.mobi

HTML:
https://progit2.s3.amazonaws.com/en/2016-03-22-f3531/progit-en.1084.zip


Regards,
Graeme

[Bo Berglund]

On Wed, 31 May 2017 01:03:54 +0100, Graeme Geldenhuys
<***@geldenhuys.co.uk> wrote:

Graeme, I am very grateful for your contribution to this discussion!
Aplogies if my postings are a bit longish, but I try to convey my
concerns and ignorance...

Post by Graeme Geldenhuys

Post by Bo Berglund
So this should not be created inside some user home then?

You can create Git repositories wherever you want and have read/write
access. My "/data" path is simply by 8TB ZFS data pool, where I do all
my work and store all vital data. You can use your $HOME directory
(whatever that translates to on your OS) just as well.

In the end, if all works out well, I want to have a GIT *server* on
our company network as the main repository, acessed via the internal
network by authenticated developers only. It would be the "remote"
mentioned in several GIT how-to pages I have read.
There should be no access path to this from the Internet except for
developers with a VPN channel into the company network. But this is of
course not an Internet access path since the VPN effectively places
you inside the local LAN.
Access via http would probably be the simplest way to manage I
believe.

The Smart-HTTP server setup is mentioned in section 4.6 of the ProGIT
book:
https://git-scm.com/book/en/v2/Git-on-the-Server-Smart-HTTP

I found that Apache is already installed as part of the Raspbian
Jessie PIXEL so the install steps need not be done I assume.
However, the configuration instructions here are a bit terse so I have
not dived in there yet, need to get to know GIT better first.
But do you think it can be a working solution for us to use the web
interface? It would get a user authentication mechanism in place....

Post by Graeme Geldenhuys
If you are going to share your Git repositories stored on your system
with the public

Will never happen. We work on products that we sell and we do not
share our code...

Post by Graeme Geldenhuys
You mentioned you wanted to play around with Git and get to grips with
it - hence I suggested a local setup without the need of a RPi or some
other device.

Since it looked very much like a Linux thing it appeared to me to be
simpler if I had a Linux box to test on. And I have found that RPi3 is
pretty convenient.

Post by Graeme Geldenhuys
I always install git
from source code and compile it myself (like I do with FPC and Lazarus
too). Everything is then included - as it should be. Linux distros f*ck
everything up and split it into multiple packages. eg: git-core,
git-base, git-gui, git-subversion, git-docs etc.

I also build my FPC/Lazarus installations from svn sources since a
couple of years. But only on Linux, I have not tried it on Windows
yet.

So I really should go back to the RPi and remove the git I installed
yesterday and instead try to get the sources (via Subversion?) and
build myself then? Is it possible to get the GIT sources without GIT?

Is there some additional task to perform before running make? Like for
FPC/Lazarus I had to script the install to make sure the system was
updated with extra libraries and such before building...

Post by Graeme Geldenhuys

Post by Bo Berglund
I would very much like to have a PDF copy since I usually find that
easier to read than using on-line webpage versions of books.
Could not find the PDF though...

I just had a look. The links used to be on the Table Of Contents page,
but for some odd reason they aren't there any more. No stress, The
Internet Archive always comes to the rescue.
https://progit2.s3.amazonaws.com/en/2016-03-22-f3531/progit-en.1084.pdf

Thanks Graeme!
Now I have the PDF ready for off-line reading (500+ pages...)

I have additional questions regarding the way GIT works:
Handling of line endings
-------------------------
When woking on multiple platforms like Linux, Mac and Windows there
has always been a struggle to get source code line endings compatible
with the platform. Windows uses CR-LF, Mac uses CR and Linux uses LF
and in some cases if this is wrong the sources will not work.
In CVS this was managed by the CVS client such that it expanded line
endings to the platform standard whereas the repository always used
the Linux standard LF.
So if sources were transferred between systems via CVS all was fine.

Now I have read a bit about GIT and it really looks like it checksums
whole files with a 40 char hash, which is used as the key to the
files. This would make the exact same file (content wise) on Linux and
say Windows appear different with different hash values unless there
is some mechanism in GIT that handles this.
But I have not seen any mention of this yet...
And how does one specify the type of file to put in GIT? I want to
keep also some binaries there, but these invariably contain CR, LF and
CRLF byte combinations that MUST NOT be changed when moving between
platforms...
Of course some tools may have provisions for line ending invariance
but for example php code breaks on Apache if these are wrong.
Does Lazarus or FPC ignore line ending differences?

And lastly regarding CVS conversion:
------------------------------------
Since we have not used CVS in any complex way (but we do use the CVSNT
flavour on Windows), I assume that the repository files would be
possible to convert into the GIT format. After all each file contains
a history with commit messages, timestamps, tags and branches, which
are all OK.
But the organization may well consitute a problem since we have
several cases where multiple products (projects in GIT maybe) are
sharing common code via CVS modules. So the same files are used in
several places. They are stored outside project modules (folder
hierarchies) and brought in at checkout time into subdirectories of
the project source tree.
Will that complicate GIT conversion?

[Graeme Geldenhuys]

Post by Bo Berglund
Aplogies if my postings are a bit longish,

You are like me then. :) I try to write shorter messages, but often fail.

Post by Bo Berglund
Access via http would probably be the simplest way to manage I
believe.

I actually found Gitolite the simplest (which uses SSH and manages its
own user accounts and access) - independent of the operating system user
accounts. So I highly recommend you look into that.

The Git Daemon is also very very straightforward, but doesn’t offer much
in the form of security.

I try not to use HTTP for Git if I can - it’s the slowest protocol for
use with Git. It’s like fitting a square peg in a round hole. With a big
enough hammer it WILL fit, but not ideal. ;-)

Post by Bo Berglund
I also build my FPC/Lazarus installations from svn sources since a
couple of years. But only on Linux, I have not tried it on Windows
yet.

I use near identical scripts on both platforms. So not any more difficult.

Post by Bo Berglund
build myself then? Is it possible to get the GIT sources without GIT?

Yes, FreeBSD’s Ports system does that. It downloads a *.tar.xz archive
of the latest release, then builds it locally.

Post by Bo Berglund
When woking on multiple platforms like Linux, Mac and Windows there
has always been a struggle to get source code line endings compatible
with the platform. Windows uses CR-LF, Mac uses CR and Linux uses LF
and in some cases if this is wrong the sources will not work.

I believe the default behaviour is for each platform to have native EOL
styles, after you cloned a repository. Internally, git stores files in a
set EOL style (Unix-style I believe) on all platforms.

But some people like Linus Torvalds will tell you that technically a SCM
system should *not* modify files at all, so that what you put in is
exactly what you’ll get out. Git obviously supports this model too.

Post by Bo Berglund
Now I have read a bit about GIT and it really looks like it checksums
whole files with a 40 char hash, which is used as the key to the
files. This would make the exact same file (content wise) on Linux and
say Windows appear different with different hash values unless there
is some mechanism in GIT that handles this.

Git handles that, and there will be no difference between that same
patch being applied under Linux or Windows. The commit SHA1 should be
identical. Git magic I guess. ;-)

Post by Bo Berglund
And how does one specify the type of file to put in GIT? I want to
keep also some binaries there, but these invariably contain CR, LF and
CRLF byte combinations that MUST NOT be changed when moving between
platforms...

You can create a ".gitattributes" file in the root of your git
repository. You can then specify which files should be treated as binary
data, which diff algorithm should be used, and which files should have
specific EOL styles. See the fpGUI repository as an example.

=======================[ .gitattribute ]=======================
# Set default behaviour, in case users don't have core.autocrlf set.
* text=auto

# Explicitly set some EOL styles and preferred diff style
*.pas text diff=pascal
*.lpr text diff=pascal
*.pp text diff=pascal
*.inc text diff=pascal
*.txt text
*.xpm text
*.ods diff=odf
*.odt diff=odf
*.odp diff=odf

# declare files that must always have specific EOL style
*.dez text eol=crlf

# True binary files that should not be modified (just for safety sake)
*.jpg binary
*.png binary
*.bmp binary
*.inf binary
*.INF binary
*.zip binary
*.res binary
*.tar.gz binary
*.wav binary
*.pdf binary

===============================================================

Post by Bo Berglund
Since we have not used CVS in any complex way (but we do use the CVSNT
flavour on Windows), I assume that the repository files would be
possible to convert into the GIT format. After all each file contains
a history with commit messages, timestamps, tags and branches, which
are all OK.

I haven't used CVS in over 20 years easily. I know there is a CVS-to-Git
conversion support, just like there is for SubVersion, Mercurial and
Perforce, but I've only [personally] done SubVersion conversions.

I unfortunately don't know the details of what you ask. I'm sure a
Internet search or the Git man pages will help you further.


Regards,
Graeme

[Bo Berglund]

On Wed, 31 May 2017 10:58:29 +0200, Bo Berglund

Post by Bo Berglund

Post by Graeme Geldenhuys
I always install git
from source code and compile it myself (like I do with FPC and Lazarus
too). Everything is then included - as it should be. Linux distros f*ck
everything up and split it into multiple packages. eg: git-core,
git-base, git-gui, git-subversion, git-docs etc.

I also build my FPC/Lazarus installations from svn sources since a
couple of years. But only on Linux, I have not tried it on Windows
yet.
So I really should go back to the RPi and remove the git I installed
yesterday and instead try to get the sources (via Subversion?) and
build myself then? Is it possible to get the GIT sources without GIT?

Stupid question by me...
I should use git to get git sources *before* I uninstall the apt-get
version of git. The solution is in page:
https://git-scm.com/downloads
as this command:
git clone https://github.com/git/git

I used it and it finished successfully in a rather short time.

What I received is the latest *development* version, but if I got the
idea behind GIT I should now have a git repo locally on my RPi3 and
should be able to retrieve the latest released version of git from
that, right?
If so how do I move forward and find the release, update to it and
then build and install git from the sources?

The apt-get version of the git I use now is:
~/ $git --version
git version 2.1.4

Seems a bit oldish so I updated the RPi3 according to the ProGIT book
section 1.5:

sudo apt-get install dh-autoreconf libcurl4-gnutls-dev libexpat1-dev
gettext libz-dev libssl-dev

Then tried to make as per the INSTALL file. Ran for some time and
succeeded.
So I then also got requirements for the other stuff:

sudo apt-get install asciidoc xmlto docbook2x getopt

But it failed with this error:
E: Unable to locate package getopt

so I removed getopt from the install and all other modules installed
fine.

Then I did:
make configure
./configure
make all doc info
make install install-doc install-html install-info

The last line failed as follows:

$ make install install-doc install-html install-info
SUBDIR git-gui
SUBDIR gitk-git
SUBDIR perl
SUBDIR templates
install -d -m 755 '/usr/local/bin'
install: cannot change permissions of ‘/usr/local/bin’: Operation not
permitted
Makefile:2408: recipe for target 'install' failed
make: *** [install] Error 1

Is it not påossible to put docs and info below the ~/ dir?
According to the INSTALL file on the subject of Git install it says:

Git installation

Normally you can just do "make" followed by "make install", and that
will install the git programs in your own ~/bin/ directory.

This is exactly what I would like to do instead of putting it below
/usr

What do I do next?

[Graeme Geldenhuys]

Post by Bo Berglund
~/ $git --version
git version 2.1.4

Yeah, that is a rather old version. The latest is v2.13.

$ git tag

will show all tagged releases. Just look for the latest one. You can
also use 'gitk --all' and see what the "next" branch follows on from. In
the Git project, the "next" branch is what will become the next release.

So simple checkout "next" or if you want a mare stable released version,
checkout the v2.13 release.

git checkout v2.13.0

(git will mentioned you have a detached HEAD, but that doesn't matter,
it just means you don't have a local branch that matches that release
commit.)

Post by Bo Berglund
Is it not påossible to put docs and info below the ~/ dir?

Yes, it works like most Makefile based environments - and is also
mentioned in the INSTALL file.... You need to specify the install prefix
directory.

$ make prefix=/home/<user>/

Or whatever you want the install prefix to be.

As for building the Git documentation. Yes, that has extra dependencies,
and from memory takes quite long to build. But the process is very
similar to building the Git binary itself. All the instructions are in
the INSTALL file.


Regards,
Graeme

[Bo Berglund]

On Wed, 31 May 2017 18:04:28 +0100, Graeme Geldenhuys

Post by Graeme Geldenhuys

Is it not possible to put docs and info below the ~/ dir?

Yes, it works like most Makefile based environments - and is also
mentioned in the INSTALL file.... You need to specify the install prefix
directory.
$ make prefix=/home/<user>/
Or whatever you want the install prefix to be.

I tried several times with a prefix entered first by doing this:
$ ./configure --prefix=/home/pi/
$ make install

and then just
$ make install --prefix=/home/pi/

Both errored out because they could not write to /usr somewhere...

So then I bit the bullet and used sudo:
$ sudo make install --prefix=/home/pi/
....
done && \
./check_bindir "z$bindir" "z$execdir" "$bindir/git-add"

So it seemed like it succeeded, ut why must I use sudo when the target
should be inside the pi/home dir?

Then I tested what I got:
$ which git
/home/pi/bin/git
So far so good, it seems like my git is first on path. :)

Next:
$ git --version
git version 2.1.4

Not so good because the new version should be way higher!
But look:
$ ~/bin/git --version
git version 2.13.0.311.g0339965

So specifically executing git from the user's bin dir gets the correct
version while just running it via path gets the wrong value!
How come?
Even after I did this:
$ sudo apt-get remove git-all

the response to git --version is still
2.1.4

From where does this come???
Is there another git installed on the system? I did not check this
before I installed git via apt-get.

Just for clarity:

$ $PATH
-bash:
/home/pi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games:
No such file or directory

I suspect that games are not there...

I don't understand this, which git will I execute in this situation
when I use git on the command line without the full path?

scratch, scratch, scratch....

[w***@windstream.net]

Post by Bo Berglund
the response to git --version is still
2.1.4
From where does this come???

at the command line, type in "which git" without the double quotes...

[Bo Berglund]

Post by w***@windstream.net

Post by Bo Berglund
the response to git --version is still
2.1.4
From where does this come???

at the command line, type in "which git" without the double quotes...

$ which git
/home/pi/bin/git

This is the version I have built myself from sources and it is 2.13
I installed it into the user space rather than into /usr/bin/

However if I do:
$ git --version
git version 2.1.4


But if I do:
$ /home/pi/bin/git --version
git version 2.13.0.311.g0339965

Do you now understand my confusion?
Executing git on the command line seems to execute the /usr/bin/git
instead of the first git on PATH! /home/pi/bin is first on $PATH

Seems like git is started in /usr/bin and there it is getting the
erroneous version information....
Why?

[Bo Berglund]

On Wed, 31 May 2017 22:23:30 +0200, Bo Berglund

Post by w***@windstream.net
at the command line, type in "which git" without the double quotes...

Now I made a test by looking at another RPi3 with Raspbian Jessie
where I have definitely NOT been doing anything git.
Surprise!

$ which git
/usr/bin/git

and

$ git --version
git version 2.1.4

So I went to another RPi2 I also have running Raspbian (older):
$ which git
/usr/bin/git
$ git --version
git version 1.7.10.4

This means that one does *not* have to install git on Raspbian because
it is already there....

Now remains to find out how to disable the old version shipped with
Raspbian and use the self-compiled newer version.
The $PATH location seems not to help.

[Bo Berglund]

On Wed, 31 May 2017 23:04:02 +0200, Bo Berglund

Post by Bo Berglund
Now remains to find out how to disable the old version shipped with
Raspbian and use the self-compiled newer version.
The $PATH location seems not to help.

Problem solved but not understanding why...
I logged off the PuTTY session and then back in again and now the
command returns the correct value:

$ git --version
git version 2.13.0.311.g0339965

Even though it now works it would be intersting to know *why* I had to
log off and back on for it to use the path ...

[Graeme Geldenhuys]

Post by Bo Berglund
Even though it now works it would be intersting to know *why* I had to
log off and back on for it to use the path ...

It seems quite clearly that the PATH doesn't fully take affect (in all
sessions - it definitely doesn't in existing open sessions) until you
log out and then back in. Knowing *nix systems, there must be a way to
reinitialise your already open sessions with the latest config info -
but it escapes me now on how that is done.

Regards,
Graeme

[Bo Berglund]

Continuing with GIT...

I am now studying a few on-line documents outlining the conversion of
a CVS repository to GIT. There is a script suite named cvs2svn with a
number of specific scripts for various targets like cvs2git, so I
think I can use this. But I think I have found a problem, which may or
may not exist, but I need to check:

We use the CVS modules concept a lot in our CVS repository. It
virtualizes programming projects by collecting files from various
"common" modules into a target structure for the project itself.
We have as example a CommonFiles module, which contains 26 files
implementing various commonly needed functions.
In any given actual project the specific source files for that project
is stored as a named module of files.

Now the module definition in CVS collects the files and folders from
various places into the structure of the project itself, often placing
the files for the actual project into a "source" folder. The common
files are placed in separate folders next to the source folder in the
checkout.
The net effect is that we have these specific and common files
collected into folders in the work project folder ready to be worked
on.

The beauty of this is that we do not need to duplicate common
functionality between projects as would be the case if we *copied* the
common files into the source folder. These files are used in many
places but versioned in a single place on the server.

What would be the corresponding GIT method for this?
Or is it even possible?

[Graeme Geldenhuys]

Post by Bo Berglund
The beauty of this is that we do not need to duplicate common
functionality between projects as would be the case if we *copied* the
common files into the source folder. These files are used in many
places but versioned in a single place on the server.

Yes, Git has that functionality too. It is called "submodules". Common
code can live and be maintained in it's own repository. Other
repositories can than link to that repository.

https://git-scm.com/book/en/v2/Git-Tools-Submodules

https://git-scm.com/docs/git-submodule

I have used this often. If you clone a repository that makes use of
submodules, all the information is already there (stored inside the
repository).

The first time you clone such a repository, it will not automatically
fetch the files from the submodule. You need to run (only once) the 'git
submodule init' command. Then every time you want to update the
"common/shared code" you run 'git submodule update'.

The Pro Git chapter explains it very well, and it is actually quite
simple to setup and use.

Regards,
Graeme

[Bo Berglund]

On Fri, 2 Jun 2017 10:19:49 +0100, Graeme Geldenhuys

Post by Graeme Geldenhuys

Post by Bo Berglund
The beauty of this is that we do not need to duplicate common
functionality between projects as would be the case if we *copied* the
common files into the source folder. These files are used in many
places but versioned in a single place on the server.

Yes, Git has that functionality too. It is called "submodules". Common
code can live and be maintained in it's own repository. Other
repositories can than link to that repository.
https://git-scm.com/book/en/v2/Git-Tools-Submodules
https://git-scm.com/docs/git-submodule
I have used this often. If you clone a repository that makes use of
submodules, all the information is already there (stored inside the
repository).
The first time you clone such a repository, it will not automatically
fetch the files from the submodule. You need to run (only once) the 'git
submodule init' command. Then every time you want to update the
"common/shared code" you run 'git submodule update'.
The Pro Git chapter explains it very well, and it is actually quite
simple to setup and use.

Thanks a lot, found the chapter and will be going over it this
week-end with my son-in-law who uses GIT himself.
He did not know of the possibilities before, though. He recently
converted his SVN repo to GIT. But I think he has not used this
feature.
He also earlier told me to look at GIT as replacement for CVS.

[Bo Berglund]

Report on progress:
I have found a way to convert CVS to GIT using the cvs2svn module,
which contains a cvs2git file. It was described in this article:
https://devsector.wordpress.com/2014/05/17/migrate-cvs-to-git-with-cvs2svn/

Since our CVS server resides on Windows and is driven by CVSNT and the
conversion process (a Python system) uses the cvs executable to
extract revision information from the version files I decided to give
it a try on Windows7.
After some misunderstandings causing me to restart the conversions I
finally managed to convert some test projects.

But there is one very strange thing happening:
After the conversion is done according to the steps in the article
this happens when I run a check on the created GIT repo:

$ git status
On branch master
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)

deleted: SSM1/ReadMe.txt
<then follows a list of ALL files in the project all marked deleted>

Why is the repo in this condition?
And what to do about it? Obviously going over it with
git reset HEAD <file>...
isn't really practical since this example project (an Android App)
contains deeply nested folder trees with hundreds of files.

I am worrying that I am doing this all wrong...

It is possible to clone the repo and get the expected file tree out,
though.

Another issue is that it seems like the conversion of a CVS repository
consisting of many different modules ("projects" if you will) ends up
with a single huge GIT repo which can only be cloned in its
entirety...
How can one store multiple and *separate* projects in GIT?

[Graeme Geldenhuys]

Post by Bo Berglund
Why is the repo in this condition?
And what to do about it? Obviously going over it with
git reset HEAD <file>...
isn't really practical since this example project (an Android App)
contains deeply nested folder trees with hundreds of files.

Like I said, I have no experience with CVS or covertions from CVS-to-Git.

1) I suggest you start with a 'gitk --all' and see how the history
of your repository looks like, and if there is any history at all.

2) You don't have to reset each file one by one. You can do it all
in one go with: git reset --hard

But why all your files are marked for deletion is a mystery to
me.

3) This is never mentioned in ANY repository conversion guides, but
something I ALWAYS do. After you ran your automated migration to
Git, I would use a comparison tool (eg: Beyond Compare) and
compare a _clean_ original repository against the newly migrated
Git repository. Do a new checkout of the original repository (CVS,
Subversion etc) to make sure it is clean and you know it is the
latest code from the server. Technically there should be NO
differences, but on the small chance that there is, now is the
time to see the differences and commit those into Git so you know
going forward both repositories have the exact same state (at the
time of migration).

Regards,
Graeme

[Bo Berglund]

On Mon, 5 Jun 2017 15:46:43 +0100, Graeme Geldenhuys

Post by Graeme Geldenhuys
1) I suggest you start with a 'gitk --all' and see how the history
of your repository looks like, and if there is any history at all.

The history is there all right, but I think that the tags are a bit
different...
It seems like GIT does not like the fact that in CVS only changed
files can be committed thus revisions of files evolve on a file by
file basis. So when a CVS tag operation is performed the tag is
applied on all of the files in the current state. This means that a
CVS tag contains files at different revision levels and apparently
this is something GIT does not like.
So it tries to "fix" this by making fake commits or something similar.

Post by Graeme Geldenhuys
2) You don't have to reset each file one by one. You can do it all
in one go with: git reset --hard
But why all your files are marked for deletion is a mystery to
me.

I can go elsewhere and clone the creatred repository at which time I
am getting the HEAD revision checked out. Relevant files compare fine
except they are not the correct timestamp.

Concerning nested projects...
I found a caveat in the documentation for CVS2GIT:
"Only single projects can be converted at a time. Given the way git is
typically used, I don't think that this is a significant limitation."

In fact it is a limitation in our case, but I have to go looking for a
way to get around it...

[Giuliano Colla]

Post by Bo Berglund
Relevant files compare fine
except they are not the correct timestamp.

This is one of the down sides of GIT: it doesn't preserve timestamps.

As a workaround you may write a small script which will touch each file
of the newly created repository with the time stamps of the original
folders. Here enclosed a python script which I used for that purpose;
you may see if it fits your needs.

After that, if you need to preserve timestamps, you may take advantage
either of git-restore-time (which will change the timestamps to the
commit time):

https://github.com/MestreLion/git-tools

or of the metastore utilities:

https://github.com/przemoc/metastore

which adds an additional metadata file, where original timestamps are
stored, and used to properly set them at each commit, and to restore
them at each checkout, pull, etc.

Hope that it helps,

Giuliano

[Graeme Geldenhuys]

Post by Giuliano Colla
This is one of the down sides of GIT: it doesn't preserve timestamps.

You are the second person I hear say that. Why is preserving timestamps
important? What use is there for it?

Git already tracks the commit date/time, authored date/time and other
metadata - all available in the Git log. Those are git log metadata. But
Git really only tracks file content changes over time - that is the
important bit everybody needs. Git doesn't even track directories - only
file content changes are tracked, but even with only that information,
Git is clever enough to track file renames and movement of files between
locations in the repository.

I honestly see no need for timestamps. If mean for things like a
compiler should detect when files are changes - well, compilers should
check for the content of file changes (via checksums or CRC etc).
Timestamps can be spoofed or changed without the contents of the file
changing - thus totally pointless. Then there is things like daylight
savings time, timestamps across timezones etc. All things that could
affect timestamps, but not the content of a file. So Git is right by not
bothering with file timestamps - query the git-log instead.

Regards,
Graeme

[Giuliano Colla]

Post by Graeme Geldenhuys
You are the second person I hear say that. Why is preserving
timestamps important? What use is there for it?

One of the important things of programming, both for economical reasons
and for better quality, is to be able to reuse code modules. Timestamps
provide you the ability to recognise at a glance which modules are
unchanged, old, thoroughly tested ones, and which ones are newer or
modified. No GIT command is as fast as ls -lrt *.c or ls -lrt *.pas.
Moreover, once a program is deployed, a preserved timestamp provides an
intrinsic automatic version control. Again faster and safer than any
tag, which requires manual intervention, and can be error prone.
If you google for git timestamp or git metadata, you'll see how the
matter is considered important by quite a number of people.

Giuliano

_______________________________________________
fpc-other maillist - fpc-***@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-

[Graeme Geldenhuys]

Post by Giuliano Colla
One of the important things of programming, both for economical reasons
and for better quality, is to be able to reuse code modules.

Strange that in my 25+ years of programming I have never needed that.
Somehow I always managed just fine, and relied or more reliable sources
of information for version control and reusable code modules.

Regards,
Graeme

[Giuliano Colla]

Post by Graeme Geldenhuys
Strange that in my 25+ years of programming I have never needed that.
Somehow I always managed just fine, and relied or more reliable
sources of information for version control and reusable code modules.

Well each one selects its way of working, depending both on personal
taste, and on the specific requirement of the applications.

I made a quick check taking for reference a data entry module. It's been
used in 220 different projects from 2006 up to now. In many cases it's
just a vanilla version, which was submitted to a number of improvements
along the years, while in some cases it has some special features
added, to accommodate some specific peculiarities. Timestamps tell the
history, while the file size indicates the presence of special features.

Now, if I must face project 221, I will pick up as a starting point the
most similar project which might be five years old, knowing that I'll be
able to reuse a good quantity of its modules. But if I clone the GIT
repository in a new one for the new project, and I lose the timestamps,
I lose all the information which makes it easy to tell apart what I can
keep as it is from what is worth updating with the improvements made in
later project. Of course I will write from scratch what makes this
project different from the previous one, and again, at a later time,
timestamps will tell me at a glance what was inherited from a previous
project and what is new.

Giuliano

_______________________________________________
fpc-other maillist - fpc-***@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailm

[w***@windstream.net]

Post by Giuliano Colla
One of the important things of programming, both for economical reasons
and for better quality, is to be able to reuse code modules.

Strange that in my 25+ years of programming I have never needed that. Somehow
I always managed just fine, and relied or more reliable sources of
information for version control and reusable code modules.

as an old coder who only recently started to just barely dabble with version
control systems, timestamps are/were extremely important for the reasons others
have given... mainly being able to tell code libraries age and how well tested
they have been since they were written and put into use...

FWIW: i still don't fully understand VCSes and find myself, more times than not,
copying a code file to another name to save its current form before modifying it...

[Bo Berglund]

On Tue, 6 Jun 2017 12:30:58 +0200, Giuliano Colla

Post by Giuliano Colla

Post by Bo Berglund
Relevant files compare fine
except they are not the correct timestamp.

This is one of the down sides of GIT: it doesn't preserve timestamps.
As a workaround you may write a small script which will touch each file
of the newly created repository with the time stamps of the original
folders. Here enclosed a python script which I used for that purpose;
you may see if it fits your needs.
After that, if you need to preserve timestamps, you may take advantage
either of git-restore-time (which will change the timestamps to the
https://github.com/MestreLion/git-tools
https://github.com/przemoc/metastore
which adds an additional metadata file, where original timestamps are
stored, and used to properly set them at each commit, and to restore
them at each checkout, pull, etc.

Thanks, but I'm not that paranoid about time stamps. It really was
just an observation.

[Graeme Geldenhuys]

Post by Bo Berglund
It seems like GIT does not like the fact that in CVS only changed
files can be committed thus revisions of files evolve on a file by
file basis. So when a CVS tag operation is performed the tag is
applied on all of the files in the current state. This means that a
CVS tag contains files at different revision levels and apparently
this is something GIT does not like.

I think Git is trying to make something good out of a bad situation. I
wouldn’t blame git if it didn’t get things 100%. The way CVS and
TeamCoherence (which was an improvement over CVS too) all applied
revisions per file only, instead of a whole “repository state snapshot”.
That behaviour was very pre-historic, [and personally] a terrible
design. You could never get a decent snapshot of all files in the
repository at a specific point in time. That was one of the major new
features and improvements that even SubVersion introduced.

Post by Bo Berglund
So it tries to "fix" this by making fake commits or something similar.

I guess it’s trying to fix a “hack” with a hack. ;-)

Post by Bo Berglund
I can go elsewhere and clone the created repository at which time I
am getting the HEAD revision checked out. Relevant files compare fine

Perfect.

Post by Bo Berglund
except they are not the correct timestamp.

Git only tracks file contents, and some extra commit related metadata.
Timestamps are really irrelevant when it comes to managing file content
changes over time.

Regards,
Graeme

[Bo Berglund]

On Tue, 6 Jun 2017 23:25:03 +0100, Graeme Geldenhuys

Post by Bo Berglund
So it tries to "fix" this by making fake commits or something similar.

I guess it’s trying to fix a “hack” with a hack. ;-)

I will have to get into understanding GIT operations more so I can see
the existing tags, right now I am only seeing the output of the
conversion process.
Have to read the manual...
Maybe the extra tags are all gone by now?

Post by Bo Berglund
I can go elsewhere and clone the created repository at which time I
am getting the HEAD revision checked out. Relevant files compare fine

Perfect.

Yes, it seems like at least HEAD is OK. :)

For a very simple project that is, I will have a look at the Android
App project later, it is not my baby but it is huge in comparison to
the Pascal projects I usually work on both concerning number of files
tracked and file system directory depth...

Post by Bo Berglund
except they are not the correct timestamp.

Git only tracks file contents, and some extra commit related metadata.
Timestamps are really irrelevant when it comes to managing file content
changes over time.

Well, CVS(NT) handles the timestamps as follows:
- At commit it stores the UTC time of the commit with the file in the
repository. Note that it is not the filesystem time stamp.

- At checkout it applies these timestamps to the local files so it is
possible to get a picture of the state at commit time wise. The time
is set depending on the state of the client computer regarding time
zone, DST etc.

- At updates (when changes made by other developers are retrieved) the
file is stamped with the local timoe of update. This is done to insure
that any make process following will recompile the file, it is just as
if the local developer had modified the file.

What file gets compiled at build time depends entirely on how the
development environment is thinking, i.e. make compares the object
file and source fiile time stamps for this and some IDE:s with
built-in intelligence do the same I assume.

So time stamp handling is not that important really it's just that I
observed the difference when comparing the GIT version of a project
with what I had in my CVS sandbox. And it only applies to a clean
checkout in CVS, otherwise the current time is what one gets.

Anyway, make clean usually is what is initially used after getting
other people's contributions, right? For C projects at least.

I don't know how Lazarus handles the selection of files to compile,
though. Or is it an FPC determination?
Could be either the compiler or the development IDE handling that...

[Graeme Geldenhuys]

Post by Bo Berglund
So time stamp handling is not that important really it's just that I
observed the difference when comparing the GIT version of a project

Fair enough and a good observation.

Git stores such "metadata" like author name + email, authored timestamp,
committer name + email, commit timestamp, commit message etc in the Git
repository. File systems alone obviously can't handle such information.
You can obviously query the Git repository for that information and
because Git history is all local, it is very fast to retrieve - unlike
SubVersion, TeamCoherence, CVS etc. that must query a remote server over
TCP/IP.

Regards,
Graeme

[Lukasz Sokol]

Post by Bo Berglund
I have found a way to convert CVS to GIT using the cvs2svn module,
https://devsector.wordpress.com/2014/05/17/migrate-cvs-to-git-with-cvs2svn/

[...]

Post by Bo Berglund
I am worrying that I am doing this all wrong...
It is possible to clone the repo and get the expected file tree out,
though.

You have (probably) created the repo in 'bare' mode (no checked out working copy?)

(or the conversion process does it inadvertently)

When you clone, you get the working directory, then you'd need to push changes to bare repo
(just like you would on SVN or CVS by default)

(hope it's that, idk because I don't use bare repos).

-L.

_______________________________________________
fpc-other maillist - fpc-***@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinf

[Lukasz Sokol]

Post by Graeme Geldenhuys

Post by Bo Berglund
Since my local system is a Windows 7 laptop I have to resort to an RPi
to get the Linux system for which the commands are native...

Git is native on all supported platforms now.

Post by Bo Berglund

Post by Graeme Geldenhuys
mkdir /data/myremote.git
cd /data/myremote.git
git init --share --bare .

When you install Git under Windows using the official installer, it
includes Bash Shell integration and a shortcut on the desktop. So the
above commands will work as-is on Windows too.

TortoiseGIT also lets you create a 'bare' repo.

[...]

Post by Graeme Geldenhuys
Regards,
Graeme

-L.

_______________________________________________
fpc-other maillist - fpc-***@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman

[Graeme Geldenhuys]

Post by Lukasz Sokol
TortoiseGIT also lets you create a 'bare' repo.

I don't disputed that, and I don't mind others using gui front-ends to
git - as long as they know they are seriously limiting their abilities
and functionality of Git.

I have reviewed a lot of Git GUI front-ends in depth over the years. In
every single case they lagged behind Git command line interface features
or couldn't do some of the more advanced functionality. And more often
than not they got confused about the state of the git repository. They
were also always slower to use than the command line interface.

Regards,
Graeme

[Lukasz Sokol]

Post by Graeme Geldenhuys

Post by Lukasz Sokol
TortoiseGIT also lets you create a 'bare' repo.

I don't disputed that, and I don't mind others using gui front-ends
to git - as long as they know they are seriously limiting their
abilities and functionality of Git.

It wasn't my intention to 'dispute' anything, only just to point that out. :)

Post by Graeme Geldenhuys
I have reviewed a lot of Git GUI front-ends in depth over the years.
In every single case they lagged behind Git command line interface
features or couldn't do some of the more advanced functionality. And
more often than not they got confused about the state of the git
repository. They were also always slower to use than the command line
interface.

I have TortoiseGIT 1.8.16.0 with git 2.8.2.windows.1... on winxp it's the last
revision I can use (starting with TGit 2.0 winxp support is removed)

But, it's (or, it 'Feels') much faster.

At least compared to TSVN 1.6.5 with SVN 1.6.5 i also have on that machine.
(plus, this needs a special area to hold the 'bare'/repo 'remote' directories, or a server,
which git doesn't need that badly)

As per features, I'm probably not even using 5% of them... but it lets me
get used to having them, in case I need them later on;

I remember looking at TGit when it was not yet 'on par' with git's own features
(pre-1.0 TGit) and it wasn't as impressive as it is now.

And having just tried that-era (2.8.2.windows.1) git-bash, on my winxp sp3,
I can't say I'm impressed with its speed... even ls -l is a 1-2s delay...
(that may or may not apply to git itself though, it just discourages me from trying it).

Post by Graeme Geldenhuys
Regards, Graeme

Kind Regards
-L.

_______________________________________________
fpc-other maillist - fpc-***@lists.freepascal.org
http://lists.f

[Graeme Geldenhuys]

I'd use something like an HP
Microserver with at least mirrored discs,

Yes the HP Microservers are excellent! I highly recommend them, and they
are cheap as chips - so a real bargain. I would load it with good
NAT/Server style disks (eg: Western Digital Red disks) and a small SSD
boot disk (or even a USB stick). Run FreeBSD or Linux with ZFS in
RAID-z1 or RAID-z2. Like I said is some other thread, I wouldn't trust
my data on any other file system again - only ZFS for me! A all-in-one
easy to set up system is FreeNAS (which comes standard with ZFS) and
includes other since things like easy directory sharing via FTP, HTTP,
NFS etc. All managed via a simple web interface.



Regards,
Graeme

[Dimitrios Chr. Ioannidis via fpc-other]

Hi,

Run FreeBSD or Linux with ZFS in RAID-z1 or RAID-z2.

What a coincidence, we scheduled to test OmniOS with napp-it in a HP
MicroServer . ;)

https://www.napp-it.org/downloads/omnios_en.html .


regards,

[Graeme Geldenhuys]

Post by Bo Berglund
Is there a good way to set up an RPi3 box as a GIT server and get
going with that?
Can GIT work in a way that would be comparable to CVS regarding
concurrent development etc?
Any suggestions on where to start?

Git doesn't require a "server" - there is no "git server" like you find
with CVS or SubVersion. Gitolite is a very fast and simple solution to
get a shared Git environment up and running in the traditional
client/server model that CVS and SubVersion users are used to. A true
distributed environment will have no single "server location" of the
source code.

There is a Git Daemon that can be run on Unix or Windows systems. But
this is in essence just to supply you with access to a Git repository
via the much more efficient git:// protocol, instead of abusing the
http:// protocol. The recommended way to access a Git repo is via SSH
(which Gitolite also uses). You supply you public key, and don't have to
bother with usernames and weak passwords.



Regards,
Graeme

[Giuliano Colla]

Post by Bo Berglund
Any suggestions on where to start?

My suggestion would be to use http://rogerdudler.github.io/git-guide/ as
a quick reference just to have at hand the syntax of the most used
commands, together with the links to the full documentation.

IMO a new user should be aware from the beginning of the usage of the
.gitignore file, to tell what you don't want to track with GIT. Failing
to use it, you may have your logs cluttered with bak files you don't
care about, maybe object files you'd rather rebuild if such is the case,
and so on.

A thing that I find very annoying with Git is that when you clone a
repository, or you fetch from a repository, the dates of the files
aren't preserved, which makes it impossible to tell apart at a glance
what has been recently changed from what is old.
If this is important for you, you have at least two tools available:

* the MestreLion git-tools (https://github.com/MestreLion/git-tools)
providing git-restore-mtime which will change the time-stamp of all
files to their commit time
* The Metastore utility which you may get from
https://github.com/przemoc/metastore. It adds a .metadata file in
your repository, with the original datestamps of your files which
are updated at ech commit, and it uses to restore the proper date at
each checkout/pull request, provided you add the post-checkout and
pre-commit scripts (which are supplied in the examples folder) in
the .git/hooks/folder of each user.

I agree with others that an HP Microserver would be a very good choice,
as an alternative (or as an addition) to a Linux VM in a Windows
machine. Never put all your eggs on the same basket! Also an external
USB HD should be considered as an inexpensive security backup.

Giuliano