Chapter 1: Prerequisites

Long story short, Heiya runs on Python 3, so at a bare minimum you must have Python installed on your computer. But there are also some optional things to make your life easier.

1.1 - (Optional but strongly recommended) Install Anaconda

Note: If you already have python, jupyter_notebook, and pyperclip installed on your computer, feel free to skip this section.

In order to use it more easily, you might want to optionally install jupyter_notebook.

Configuring python and jupyter_notebook might be a daunting task for those who are not familiar with using Python. Therefore, it would be much better if there is something that just takes care of everything automatically and not having to mess around with the code. That is what Anaconda does.

For the majority of people, you will want to just install Anaconda from their Official Website (Link).

Note I have no affiliation with Anaconda, it just makes setup and install python prerequisites much easier. After you have Anaconda installed on your computer, it should already configured pip, jupyter_notebook, and python for you, and if you don't know what I am talking about, don't worry about it and just restart your computer.

Of course, if you are familiar with python configuration, you can do it on your own, Anaconda is optional.

1.2 - Install Heiya (Mandatory)

Installation of Heiya should be fairly straight forward if you did the previous section or already have everything setup.

Note: If you used Anaconda, run the following in Anaconda Prompt (Help in 1.2.2):

                
                  conda activate base
                
              
                
                  pip install heiya
                
              

Once finished, it should look something like below.

If you configured on your own, simply run the following command in Terminal with administrator privileges (Help in 1.2.1).

                
                  pip install heiya
                
              

1.2.1 - Troubleshooting

If you encounter some error code, most likely one of two things is going wrong:

  • 1. You don't have pip installed on your computer. To solve this, run the following command to install pip.
                        
                        curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
                        python get-pip.py
                        
                      
  • 2. You don't have the Admin permission to run the commands. You are most likely going to see something like:
                        
                          ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'c:\\python310\\scripts\\pip.exe'
                        
                      
    Just make sure you are running Terminal using Administrator privilidges.
  • 1.2.2 - How to run Terminal with administrator priviledges?

    macOS

    On macOS, it is pretty straight forward. Open Launchpad and search for "Terminal", and open that app.

    Type in the above command, and press enter. Terminal will prompt you to enter the password of your computer, just do it and it is complete.

    Windows

    On Windows 11, there are many ways to run Terminal (Or Powershell/Command Prompt if you are on Windows 10 or below, it is the same thing).

    Simply right click the Windows icon on the bottom left of your screen, and press the option to Run Terminal (or Powershell) as Admin.

    1.2.2 - How to open Anaconda Prompt?

    Open Anaconda Navigator, look for something that says "Anaconda Prompt/Powershell Prompt/CMD.exe Prompt". Basically, anything with the name "Prompt" will probably do.

    Press "Launch".

    Chapter 2: Get Started

    For the most common usage, assuming you are a photographer that has some JPGs (either you shoot JPG or export from Lightroom as JPG), and you want to convert them to HEIF or AVIF so you can import them into your iCloud/Google Photos so these photos take very little storage. Here's how to quickly get yourself started.

    Note: This looks long because is a babysit tutorial that covers in great detail. Once you get it, it is brainless to run.

    2.1 - Open Jupyter Notebook

    First, open Anaconda Navigator App on your computer (Assuming you already have it installed), and open "Jupyter Notebook" inside the app. If you did not do the optional stuff, I assume you know what you are doing and just open a Jupyter Notebook page.

    A webpage should open. Navigate to a place where you want to save this tool, and click "New - Notebook Python 3" on the top right of the Jupyter Notebook web page.

    A new webpage that looks something like this should appear. First click "Untitled" to name it to something you like, for example, "jpg_hei". In the below right after "In [ ]: 1" is the place where you can enter some code.

    2.2 - Make the Tool

    Copy all the code and paste them into your newly created notebook. This code is also available on Github in pipeline.ipynb (Link).

                  
                    """
                    JPG to HEIF/AVIF
                    Convert JPG shoot by camera to High Efficiency Images with metadata to import to iCloud photo library.
    
                    import heiya
                    import pyperclip as clip
    
                    # Custom file path override, usually just leave it blank if you use pyperclip
                    source_dir = ""
    
                    # Get the image location directly from the clipboard
                    if source_dir == "":
                        source_dir = clip.paste()
                        
                    # This will convert all the JPG in source_dir to HEI.
                    # source_format = 0 -> JPG. (You don't need to change this for most of the time, you can also set to 1 for PNG.)
                    # target_format = 0 -> AVIF (For smaller file size but less compatible, requires iOS 16/macOS Ventura or higher).
                    # target_format = 1 -> HEIF (For best compatibility but slightly larger file size, requires additional extension to open on Windows).
                    heiya.to_hei.convert_image_in_dir_to_hei(source_dir, source_format=0, target_format=1)
                  
                

    Your notebook should already be saved, if not, do a quick "File - Save As" on the Notebook tool bar to make sure your notebook is created.

    2.3 - Run the Code

    Now, use Finder(macOS)/File Explorer(Windows) to go to the place where you store all of the JPGs you desire to convert.

    2.3.1 - Copy the directory address.

    Copy the directory address to your clipboard.

  • Windows: File Explorer -> Go to your target folder -> Single Click address bar -> Copy the file path in address bar.
  • macOS: Finder -> Go to your target folder -> Hold Alt (File path should apper in bottom left) -> Right click on folder and copy file path.
  • At this point, you might have something similar to this example stored in your clipboard. This is a SD card from a Fujifilm camera, your address might vary depending on where your photos are.

                  
                    D:\DCIM\207_FUJI  (Windows)
                    /Volumes/SDXC/DCIM/207_FUJI  (macOS)
                  
                

    Now, you can do one of two things:

  • If you just want to use the directory in your clipboard, do nothing.
  • If you wish to enter a custom path, paste the address into source_dir.
  • 2.3.2 - Run the code.

    Make sure to select the cell by single clicking the code.

    Navigate to your tool bar, and "Cells - Run Cells" to run the newly copied code.

    Additionally, you can use Shift + Enter on your keyboard to make this operation faster.

    Heiya should automatically start to convert the JPGs, and will print the progress down below.

    Horray! Now just drag all the HEIF/AVIF files into Apple Photos/Google Photos and enjoy amazing photos that takes up very little storage!

    2.4 - Conclusion

    In this section, I hope I did a fair job detailing basic usage of this tool and accounted for any obstacles you might encounter using it.

    From now on, all you need to do is open the tool in Jupyter notebook, copy the folder path, and run the code. Should be pretty brainless.

    If you have any question, feel free to contact me by going to my contact page (Link).

    2.5 - H.264 to H.265 Video Encoding

    Many cameras nowadays shoot video in the H.264 codec, which is compatible to most devices, but the legacy encoding yields a larger file size.

    More advanced cameras can shoot H.265 codec (Some call it HEVC codec), which is a more modern codec that yields significantly smaller file size, and modern hardware and operating system now have better support for the Codec (Although you do need to pay a small amount of money to use it on Windows, Apple devices have no problems).

    With a little setup, you can transcode video using similar workflow mentioned above in the image processing section.

    2.5.1 - Install FFMPEG to System Path

    Heiya will call an video encoder called FFMPEG to transcode your videos, which is not bundled in Heiya and require manual setup.

    Fortunately, other people have already written comprehensive guides on how to do this, so please set up FFMPEG using these very helpful guides.

  • Back to the Basics: Use and Install FFmpeg, Nicholas Xuan Nguyen, TALearning, 2022.
  • 2.5.2 - Make the Tool

    Similar to the above heiya.to_hei.convert_image_in_dir_to_hei() function that will convert all the images in a directory to HEI, as of version 2.3.0, you can now convert all video in a directory to MP4 that are encoded using the H.265 codec using a similar function heiya.to_hei.convert_video_in_dir_to_h265().

    This code is also available in the demo pipeline: pipeline.ipynb

                  
                    """
                    MP4 (H264) to MP4 (H265)
                    Convert MP4/MKV (H264) to MP4(H265) using libx265 Software Encoding.
                    """
    
                    import heiya
                    import pyperclip as clip
    
                    # Custom file path override, usually just leave it blank if you use pyperclip
                    source_dir = ""
    
                    # Get the image location directly from the clipboard
                    if source_dir == "":
                        source_dir = clip.paste()
                        
                    # This will convert all the MP4/MKV in source_dir to .MP4 using H265 codec.
                    # source_format = 0 -> MP4. (You don't need to change this for most of the time, you can also set to 1 for MKV.)
                    heiya.to_hei.convert_video_in_dir_to_h265(source_dir, source_format=0, postpend="_h265")
                  
                

    Note that heiya.to_hei.convert_video_in_dir_to_h265() has a new parameter called postpend="_h265". This parameter will postpend "_h265" after the file name of the generated file. (i.e. video.MP4 -> video_h265.MP4) This is because often you will have the original file also having the ".MP4" extension and we don't want to overwrite the original.

    You can also change this parameter to anything you like, or simply "" if you don't want postpend string. Note if you don't specify this parameter, Heiya will default postpend "_h265" to the newly generated file just to play safe.

    Chapter 3: Advanced Usage

    There are several functions that are also potentially useful but not demonstrated in the guide above.

                  
                    # Convert one image to HEIF/AVIF
                    heiya.to_hei.convert_image_to_hei(source_image, target_format)
    
                    # Convert one video to H265
                    heiya.to_hei.video_to_h265(source_video, output, postpend)
    
                    # Convert AVIF/HEIF to image
                    heiya.from_hei.convert_hei_to_image(source_hei, target_format)
                    heiya.from_hei.convert_hei_in_dir_to_image(source_dir, source_format, target_format)
                    heiya.from_hei.convert_hei_in_dir_to_jpg(source_dir, source_format)
    
                    # Transfer EXIF from one image to another
                    heiya.meta_transfer.img_meta_transfer(source_image, target_image, lens_model)
                  
                

    For now, I have limited time and effort, so any advanced usage and how to use other functions of Heiya can be found in the following notebooks.

    Name Link
    Pipeline Demo (Recommended) https://github.com/wu-hongjun/heiya/blob/main/heiya_basic_demo.ipynb
    Heiya Basic Demo (Extra) https://github.com/wu-hongjun/heiya/blob/main/heiya_basic_demo.ipynb
    Heiya Full Demo (Extra) https://github.com/wu-hongjun/heiya/blob/main/heiya_full_demo.ipynb

    Chapter 4: Heiya Functions

    This chapter will be the home to document all the functions available to call independently so you can build your own pipeline.

    It is work in progress, and most information you are looking for can be found in the Advanced Usage section above.


    Chapter 5: To-Do

    Some potential improvements that can be done include:

  • Currently Heiya only supports converting to 8-bit (Which is mostly enough, since JPG can do at most 8 bit). However, enabling Conversion to 10bit(HEIF) or 12bit(AVIF) is a welcomed improvement.
  • One potential solution to enable higher bitrate is to deprecate pillow_heif and instead rewrite the entire tool using imagemagick. Although I shall note that if you have a camera that already shoots HEIF, you probably don't need this tool.
  • Some additional video encoding capabilities is under active development by wrapping ffmpeg.
  • Of course, a Heiya GUI would be nice.
  • An automatic deployment of the tool might be a welcomed addition too.

  • Chapter 6: License

    Heiya is distributed in the license in LICENSE.txt.