Johan de Koning
Note (7 augusts 2009): patch 10889 is now part of the source tree and cupcake branch is used instead of cupcake-release
Note (30 july 2009): because of the changes made inside the Android repository, I decided to change the 10501 patch so that the wlan.ko module is not extracted from the device. The wlan.ko module is part of the dream-open project or you can build the latest version of the wlan.ko from source (as described in the blog posts). Instead of patch 10501, patch 10889 is used.
Note (27 july 2009): based on this forum post from JBQ (Software Engineer at Google), I decided to take the cupcake-release branch for the Building 1.5 series. I will write a different blogpost later about the other branches
Note (15 july 2009): the platform/vendor/htc/dream project inside the Android source tree is renamed to platform/vendor/htc/dream-open. Only the local_manifest.xml and the command for applying the patch should be changed (changes are made in this blogpost). The project is renamed because of some cleanup actions by Google. Problem is now with the latest Android1.5r2 source code, that Wifi is not working. Try to found a solution for that.
Note (24 june 2009): contributed a patch to the Android Source (change 10501) for updating the extact-files.sh and AndroidBoard.mk scripts which where outdated for the proprietary files. This simplifies the process of getting the source a little bit.
Note (9 june 2009): I updated this blogpost to make wifi and the led light of your Android Dev Phone working inside your own build.
In the previous blogpost I described how to set up the build environment. The next step will be downloading the source and apply some changes so that it will succesfully build. Getting the source is described at http://source.android.com/download but I will give a more detailed overview (and the details for making the build ready for the Android Dev Phone)
Installing repo
The repo tool will be used to get the Android source from Git. Before you can use Repo it should be installed, initialized and configured. I did not described the installation of repo in setting up the build environment because the repo script is Android specific.
Open a terminal inside Ubuntu and make sure that you are inside your home directory
cd ~
Inside the home directory, create a new directory called bin
mkdir bin
The bin directory should be part of your path ($PATH). Edit the .bashrc file inside your user home directory (which is the current directoy you are working) and add the following line to the end of the file
export PATH=$PATH:~/bin
Save .bashrc and open a new terminal (this will execute .bashrc). Check if the bin directory is inside your path
echo $PATH
The results will be different for each installation because a different username will be used (or you would like my username jcdekoning) and maybe even more locations are inside your path. The result within my installation gives back
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/home/jcdekoning/bin
The last part (home/jcdekoning/bin) is pointing to the bin directory inside the users home directory (in my case /home/jcdekoning).
The next step is to download the repo script (with curl) and make it executable (chmod). Use to following two commands
curl http://android.git.kernel.org/repo > ~/bin/repo chmod a+x ~/bin/repo
Initializing the Repo client
You need to make a location where you want to store the files (source, configuration, etc) from Git. I create a directoy inside the users home which is called mydroid
cd ~ mkdir mydroid cd mydroid
Inside the mydroid directory you need to execute repo init to specify where the android files are remotely located. By running repo init -u git://android.git.kernel.org/platform/manifest.git you will get the master version of the Android source. The problem with this version is that it is mostly not stable and can have expiremental code. Therefore I use a specific branch of the Android source which hold a stable build for Android 1.5
repo init -u git://android.git.kernel.org/platform/manifest.git -b cupcake
Repo will ask for your name and email address. When you are planning to submit code back to the Android source, use a email address which is assciated with a Google account.
If repo init is succesfully finished you will receive a message with at the
repo initialized in /home/jcdekoning/mydroid
You did not download the source yet, you only configured the repo tool so that it knowns where the source can be found. Inside the mydroid directoy you will find a .repo directory
Repo configuration changes for Android Dev Phone
Note: the download of the kernel source files is only needed if you want to compile the kernel by your own. You can also skip this configuration change and use the prebuilt kernel which is part of the dream-open project. It is possible that this prebuilt kernel is not the latest version
It is possible to download the source at this point, but we need to make some changes to retrieve specific source for the Android Dev Phone (internally called Dream).
Inside the .repo directory (within mydroid) you need to create a local_manifest.xml. This file containsa description of the three extra project needed for the Dev Phone. The contents of the local_manifest.xml
<?xml version="1.0" encoding="UTF-8"?> <manifest> <project path="kernel" name="kernel/msm" revision="refs/heads/android-msm-2.6.27"/> </manifest>
With the local_manifest.xml in place it is time to get the source code. Inside the mydroid directory (and not the .repo directory) execute the command
repo sync
No sit back, relax, drink some coffee or read some of my other blog posts because downloading the code will take some time.
Proprietary binaries
Note: the patch changes are now part of the source tree.
Some closed source files which are specific for the Dream need to be extracted from a Android device running 1.5. These files are not part of the source tree but are needed for running Android on the Android Dev Phone. Within you mydroid directory you find /vendor/htc/dream-open. There is a script called extract-files.sh. This script will use the adb tool to retrieve the files from an android device. But the content of this script is outdated for version 1.5.
To update the content of the extract-files.sh script (and the AndroidBoard.mk file) get a patch (10889):
repo download platform/vendor/htc/dream-open 10889/4
You can now get the proprietary files from you Android device by connecting it to Ubuntu and execute the extract.files.sh script
./extract-files.sh
Sometimes the Android connection between my Ubuntu in VMWare is not working properly. Therefore I use a different method for extracting the files from the official Android 1.5 image. By extracting the image provided by HTC
Using unyaffs
This part is only relevant when you don’t want or not able to extract the proprietary files from an Android device. So skip this part if you already have the files extracted. These files should be downloaded to /vendor/htc/dream-open/proprietary
The Android OS makes use of a yaffs file system. You can download the unyaffs tool at http://code.google.com/p/unyaffs/ to extract the content of an Android image. There is a prebuildversion available at the Downloads section. Place the unyaffs tool inside the ~/bin directory so that the tool can be found because it is inside the PATH definition
When the unyaffs is downloaded, make sure it can be executed
cd ~/bin chmod a+x unyaffs
Official Android images provided by HTC for the Android Dev Phone can be found at http://www.htc.com/www/support/android/adp.html You will extract the system image to get the proprietary files. At the HTC download page, make sure you download the Android 1.5 system image (signed-dream_devphone_userdebug-img-148830.zip). Inside my home directory I create a new directory where the content of the zip file will be stored (~/htc). It is important to use the same directory name otherwise the script for copying the proprietary files will not work.
The downloaded system image archive contains four different .img files
- boot.img
- recovery.img
- system.img
- userdata.img
When our own build is compiled we will end up with our build Android OS which also consists of these four different image files. The system.img will be extracted using the unyaffs tool
cd ~/htc sudo ~/bin/unyaffs system.img
If the unyaffs is executed succesfully it will give a message like end of image. When looking at the content of the htc directory you will find new created files and folders (like app, bin, etc, lib, usr). The proprietary files are stored inside the etc and lib directory. Create with your favorite text editor a new file called htc-copy.sh (the location where you save this file doesn’t matter. I saved it inside ~/htc) and give it the following content.
#!/bin/sh mkdir -p ~/mydroid/vendor/htc/dream-open/proprietary cp ~/htc/bin/akmd ~/mydroid/vendor/htc/dream-open/proprietary/akmd cp ~/htc/etc/AudioFilter.csv ~/mydroid/vendor/htc/dream-open/proprietary/AudioFilter.csv cp ~/htc/etc/AudioPara4.csv ~/mydroid/vendor/htc/dream-open/proprietary/AudioPara4.csv cp ~/htc/etc/AudioPreProcess.csv ~/mydroid/vendor/htc/dream-open/proprietary/AudioPreProcess.csv cp ~/htc/etc/firmware/brf6300.bin ~/mydroid/vendor/htc/dream-open/proprietary/brf6300.bin cp ~/htc/etc/gps.conf ~/mydroid/vendor/htc/dream-open/proprietary/gps.conf cp ~/htc/etc/wifi/Fw1251r1c.bin ~/mydroid/vendor/htc/dream-open/proprietary/Fw1251r1c.bin cp ~/htc/etc/wifi/tiwlan.ini ~/mydroid/vendor/htc/dream-open/proprietary/tiwlan.ini cp ~/htc/lib/libaudioeq.so ~/mydroid/vendor/htc/dream-open/proprietary/libaudioeq.so cp ~/htc/lib/libgps.so ~/mydroid/vendor/htc/dream-open/proprietary/libgps.so cp ~/htc/lib/libhgl.so ~/mydroid/vendor/htc/dream-open/proprietary/libhgl.so cp ~/htc/lib/libhtc_acoustic.so ~/mydroid/vendor/htc/dream-open/proprietary/libhtc_acoustic.so cp ~/htc/lib/libhtc_ril.so ~/mydroid/vendor/htc/dream-open/proprietary/libhtc_ril.so cp ~/htc/lib/libjni_pinyinime.so ~/mydroid/vendor/htc/dream-open/proprietary/libjni_pinyinime.so cp ~/htc/lib/libmm-adspsvc.so ~/mydroid/vendor/htc/dream-open/proprietary/libmm-adspsvc.so cp ~/htc/lib/libOmxCore.so ~/mydroid/vendor/htc/dream-open/proprietary/libOmxCore.so cp ~/htc/lib/libOmxH264Dec.so ~/mydroid/vendor/htc/dream-open/proprietary/libOmxH264Dec.so cp ~/htc/lib/libOmxMpeg4Dec.so ~/mydroid/vendor/htc/dream-open/proprietary/libOmxMpeg4Dec.so cp ~/htc/lib/libOmxVidEnc.so ~/mydroid/vendor/htc/dream-open/proprietary/libOmxVidEnc.so cp ~/htc/lib/libopencorehw.so ~/mydroid/vendor/htc/dream-open/proprietary/libopencorehw.so cp ~/htc/lib/libpvasf.so ~/mydroid/vendor/htc/dream-open/proprietary/libpvasf.so cp ~/htc/lib/libpvasfreg.so ~/mydroid/vendor/htc/dream-open/proprietary/libpvasfreg.so cp ~/htc/lib/libqcamera.so ~/mydroid/vendor/htc/dream-open/proprietary/libqcamera.so cp ~/htc/lib/libspeech.so ~/mydroid/vendor/htc/dream-open/proprietary/libspeech.so cp ~/htc/lib/hw/lights.goldfish.so ~/mydroid/vendor/htc/dream-open/proprietary/lights.goldfish.so cp ~/htc/lib/hw/lights.msm7k.so ~/mydroid/vendor/htc/dream-open/proprietary/lights.msm7k.so cp ~/htc/lib/hw/sensors.trout.so ~/mydroid/vendor/htc/dream-open/proprietary/sensors.trout.so chmod 755 ~/mydroid/vendor/htc/dream-open/proprietary/akmd
Execute the htc-copy.sh script so that the files will be copied to the ~/mydroid/vendor/htc/dream-open location.
chmod a+x htc-copy.sh ./htc-copy.sh
You can check if the script is succesful by going to ~/mydroid/vendor/htc/dream-open/proprietary and take a look at the contents of this directory.
What’s next?
So the source is downloaded, patched for the dream device and also the proprietary binary files are in place. The next step will be configuring the build and finally, the real work, compiling the source. I will write down more information about these steps in a next blog. Feel free to give comments on this post even if those are possitive.
Blogpost in the Building Android 1.5 serie
isn't there an error at:
"adb pull /system/lib/hw/lights.msm7k.so proprietary/lighs.msm7k.so"
I extracted the official HTC Android 1.5 image and there is a lights.msm7k.so inside the lib/hw directory.
I did not try it by my own on the HTC build (to be more specific I tried it now but that's on my own build and that's one is working).
What is the error when running that specific pull command?
I presume he means the typo "lighs" instead of "lights"
Hummm... stupid typo. Fixed it. Thanks
Hi,
When you are reffering to AndroidBoard.mk, I guess you mean it's in the "~/mydroid/vendor/htc/dream" dir and not just "~/mydroid/vendor/htc", right ?
You are right. It should be ~/mydroid/vendor/htc/dream. I will change this.
FYI: changes to the extract-files.sh and AndroidBoard.mk are now done by applying patch 10501. Above user comments can be outdated according to the updated patch process.
Hi Johan -- thanks for this article. I build a kernel using these instructions, and everything went fine except that when the Dream boots, the SD card is not auto-mounted to /sdcard. It is still recognized and loaded, just not mounted. Any idea what might be causing this?
I did not mention the mount issue with the sdcard. When I flash my phone again (i am switching between own build and HTC version) to my own builded version I will look at this issue.
Great, thanks for looking into it. I'm glad to hear I'm not the only one with this problem. One possible clue -- my build identified the SD card as:
/class/mmc_host/mmc1/mmc1:9f40
The stock kernel identifies it as:
/devices/platform/msm_sdcc.2/mmc_host/mmc1/mmc1:9f40
Sorry, this was entirely my fault and I sorted it out. I was using your old local_manifest.xml, which pointed to a different version of the kernel. Now that I'm using 2.6.27 the SD card mounts just fine. Thanks for your excellent walkthrough!
Hi,
I am confused by this repo version control. I am used to using SVN where I can view the log when i am searching for a stable release.
I stumble upon ur website before finding out i should use android-1.5r2. Where can I see this branch names?
Thank you so much. Your blog helped me a lot.
I am also a SVN or Microsoft Team Foudation Server user. I use GitWeb to find out which tags are available on a project.
For example project platform/frameworks details can be found at http://android.git.kernel.org/?p=platform/frameworks/base.git;a=summary which also gives a list of tags
I hope there is also a repo command for this. But I could not find it yet.
Hi,
With the help of this blog and http://source.android.com/using-eclipse, I am now able to build the platform project on my ubuntu machine.
Now to the next step, can I use this platform project as an sdk to run the sample apps, eg. lunar lander, on my ubuntu machine? If yes could you guide me to it? I tried to run the Calculator as a java application but the error I get is "Selection does not contain a main type". Pardon me, I am pretty new to this android development and eclipse
my goal is to write my own RIL and via a serial driver, get the android project making phone calls via a GSM modem on my ubuntu machine.
Sorry this is a double post which is also at androidforums.com
Thank you.
regards,
Adrian
Hi,
Did you or anyone else try building for the HTC Magic ?
Do you know/thing I can just follow the same process, but just extract the same device-specific files from the 'real' device ?
TIA
I don't have experience with the HTC Magic. Is it even possible to flash those devices (because there is no Dev Phone version)?
Thanks a lot for this series! Very helpful to have all this info in one place..
FYI, it appears that the platform/vendor/htc/dream project has been renamed to platform/vendor/htc/dream-open . I had to change the reference in the local_manifest.xml file to get repo to pull down the sources properly..
Strangely enough, when I tried to initialize and sync the "android-1.5r2" branch yesterday, I didn't get anything except for a 900M big ".repo" directory in the working directory. I'm going to try again today.
The previously mentioned problem is solved. It was actually caused by the outdated "platform/vendor/htc/dream" in "local_manifest.xml". The sync was interrupted, and that's why only ".repo" directory exists.
Something is strange... I always get:
Lilith:mydroid Chrigi$ repo sync
Traceback (most recent call last):
File "/Volumes/android/mydroid/.repo/repo/main.py", line 235, in
_Main(sys.argv[1:])
File "/Volumes/android/mydroid/.repo/repo/main.py", line 217, in _Main
repo._Run(argv)
File "/Volumes/android/mydroid/.repo/repo/main.py", line 123, in _Run
cmd.Execute(copts, cargs)
File "/Volumes/android/mydroid/.repo/repo/subcmds/sync.py", line 195, in Execute
all = self.GetProjects(args, missing_ok=True)
File "/Volumes/android/mydroid/.repo/repo/command.py", line 63, in GetProjects
all = self.manifest.projects
File "/Volumes/android/mydroid/.repo/repo/manifest_xml.py", line 158, in projects
self._Load()
File "/Volumes/android/mydroid/.repo/repo/manifest_xml.py", line 197, in _Load
self._ParseManifest(False)
File "/Volumes/android/mydroid/.repo/repo/manifest_xml.py", line 228, in _ParseManifest
(name)
error.ManifestParseError: project kernel/common not found
And if I remove this line then I get:
Lilith:mydroid Chrigi$ repo sync
Traceback (most recent call last):
File "/Volumes/android/mydroid/.repo/repo/main.py", line 235, in
_Main(sys.argv[1:])
File "/Volumes/android/mydroid/.repo/repo/main.py", line 217, in _Main
repo._Run(argv)
File "/Volumes/android/mydroid/.repo/repo/main.py", line 123, in _Run
cmd.Execute(copts, cargs)
File "/Volumes/android/mydroid/.repo/repo/subcmds/sync.py", line 195, in Execute
all = self.GetProjects(args, missing_ok=True)
File "/Volumes/android/mydroid/.repo/repo/command.py", line 63, in GetProjects
all = self.manifest.projects
File "/Volumes/android/mydroid/.repo/repo/manifest_xml.py", line 158, in projects
self._Load()
File "/Volumes/android/mydroid/.repo/repo/manifest_xml.py", line 197, in _Load
self._ParseManifest(False)
File "/Volumes/android/mydroid/.repo/repo/manifest_xml.py", line 255, in _ParseManifest
(project.name, self.manifestFile)
error.ManifestParseError: duplicate project platform/vendor/htc/dream-open in /Volumes/android/mydroid/.repo/local_manifest.xml
Ok, I was able to fix it
JPQ has changed the dream building process so less manual work is needed. So just change the local_manifest.xml so that only this line remains:
<project path="kernel" name="kernel/msm" revision="refs/heads/android-msm-2.6.27">
and it will work flawlessly
I didn't check if:
<project path="hardware/msm7k" name="platform/hardware/msm7k" revision="refs/heads/master">
is needed or not... is it?
checked and the msm7k line also has to be deleted.
Thanks for documenting for the benefit of others.
Do you think I could transpose this for building Donut ?
I seem to get similar build issues there regarding HTC private stuff
unless (I presume) will apply close to what you outline above ?
Will look at the Donut branch later this week after I updated the Building Android 1.5 blogposts because of the repository changes
Is the msm7k also not needed for example the led light on the HTC Dream?
I will try to create a build without this project
Checked this but the msm7k project is still needed to build some hardware libs
Seems change 10889 added a wrong tab at the end of line 77, make build error.
Thanks for the reply. Found also the tab problem in change 10889 and was trying to update the changeset. Only my knowledge of repo/git is limitted. I started a discussion at http://groups.google.com/group/repo-discuss/browse_thread/thread/96c8dcaffee8cefb
The 10889 is solved. Use repo download platform/vendor/htc/dream-open 10889/4 to get the correct file
Can't wait to build Donut soon. Johan please post your Donut build results when you are ready ....
Thanks,
Anousak
Johan. Will pulling the vendor files from the APD 1.5 OTA image from the same page as the fastboot system image work too? They're inside an OTA update zip, so pulling the files is just as easy as opening (or extracting) the zip and putting them in the necessary directory. I, for example, worked it this way in windows 7:
I downloaded the OTA update for 1.5 (don't know yet if it's for r3 CRC1 or HTC is still distributing r2 CBR43), then unzipped the system folder to the desktop. I created a directory called "proprietary" also at the destkop, and then made a windows batch file based on your linux script, but replacing ~/htc with system/ and cp with copy. Then I moved the files to my Ubuntu Virtualbox Guest through the shared folder.
It's a little convoluted (I could have just downloaded the zip in the ubuntu guest and moved the files manually myself), but I added all that info if you want to update your post.
Hi Jon,
There are two kind of Android Development Phone. One is HTC dream while the other one is HTC Magic. So does the cupcake works in both phone? and is there any differences when i build kernel image for those two kind of phones?
Help!
I've got a serious problem... I ported android 1.5 successfully to the mini2440 and tried to add the google apps, but I am stuck in a loop starting the loaderanimation and launcher again and again.
It spits out some problem with binder when I am connected to the serial port...
Without google apps it boots just fine... but with google apps it keeps reloading just after booting.... Got any idea how to track down this problem?
Greets Florian
hey when i initialized repo to mydroid...
.repo directory wasnt there in the folder...
i am using ubuntu 10.04..32 bit...could that be a problem??
root@khongquai:~/Android# repo sync
Fetching projects: 100% (117/117), done.
Syncing work tree: 65% (77/117) error: revision refs/heads/android-msm-2.6.27 in kernel/msm not found
Please help me!.
When I run command repo sync.
It downed 65% and revision="refs/heads/android-msm-2.6.27 not found??
I want run google api on android 1.5 kit mini2440 but I can not.
Please help me!
Thank!
Fetching projects: 100% (118/118), done.
Syncing work tree: 65% (77/117) error: revision refs/heads/android-msm-2.6.27 in kernel/msm not found
Please help me!
Looks that the branch for the HTC Dream is obsolete and not working anymore.