Building MxNet with Apache TVM v0.11 for Android (Arm64) [PART-1]

Ubuntu 22.04 → Relay → Android Runtime

#

Apache TVM v0.15.0 is the last release that supports Relay, making it the ideal version for deploying MXNet/PyTorch models on Android. This guide walks through building TVM on Ubuntu, cross-compiling the Android runtime, exporting Relay modules, and verifying everything on device.

This guide is tested on Apache TVM v0.11.0

🎯 Goal

#

• Build TVM v0.11 host tools for Relay model conversion
• Export models into deploy_lib.so, deploy_graph.json, deploy_param.params
• Test deployment using TVM’s Android demo app or RPC

You will also need conda installation on your devices.

Prerequisites

#

• Conda
• Linux
• Android Studio
• Android NDK [LINK] , Recommended r29 (Supports Android 16KB Requirement Out of the box)
• TVM Binaries (Below)

1. Host Build (Linux)

#

1.1. Download TVM v0.11.0

#

Suggested downloading from here instead of cloning from github, as the repository have submodules which is very easy to miss.

wget https://dlcdn.apache.org/tvm/tvm-v0.11.0/apache-tvm-src-v0.11.0.tar.gz
tar -xzf apache-tvm-src-v0.11.0.tar.gz
mv apache-tvm-src-v0.11.0 tvm && cd tvm

v0.15 is the last version supporting Relay (frontend for MXNet) Later versions use Relax which drops MXNet support.

(Tested on v0.11)

1.2. Environment Setup

#

conda create -n tvm-build-venv python=3.7 && conda activate tvm-build-venv

conda install -c conda-forge \
    llvmdev=15 clang=15 clangxx=15 cmake ninja make

pip install 'numpy<1.20' decorator scipy==1.6 psutil attrs mxnet==1.5.0

sudo apt install build-essential openjdk-17-jdk maven

1.3. Building TVM on Machine Host Build

#

mkdir build && cp cmake/config.cmake build && cd build

Configure config.cmake for CPU-only build with Relay + GraphExecutor:

echo "set(CMAKE_BUILD_TYPE RelWithDebInfo)" >> config.cmake         # reasonable debug info
echo "set(USE_LLVM \"llvm-config --ignore-libllvm --link-static\")" >> config.cmake  # IR→LLVM
echo "set(HIDE_PRIVATE_SYMBOLS ON)" >> config.cmake                 # smaller .so
echo "set(USE_GRAPH_EXECUTOR ON)" >> config.cmake                   # legacy executor
# Disable GPU backends (enable later if needed)
echo "set(USE_CUDA OFF)" >> config.cmake
echo "set(USE_METAL OFF)" >> config.cmake
echo "set(USE_VULKAN OFF)" >> config.cmake
echo "set(USE_OPENCL OFF)" >> config.cmake

Then Build for host:

cmake ..
make -j4 

The number is a total cores range from 1…MAX_CORE

1.4. Python Binding & Environment Variable

#

Append to ~/.bashrc:

export TVM_HOME=~/tvm
export PYTHONPATH=$TVM_HOME/python:${PYTHONPATH}

source ~/.bashrc

Then build the JVM package:

cd ~/tvm && make jvmpkg && make jvminstall

2. Verify if it is working (OR skip to PART-2 for Android )

#

Noted that we will use the example app we downloaded from the apache-tvm earlier, we will find the apps/android_deploy

2.1. Android App Setup (Gradle)

#

• Create gradle/wrapper/gradle-wrapper.properties

  distributionUrl=https\://services.gradle.org/distributions/gradle-7.5-bin.zip
  

• Use Gradle Wrapper + Amazon Corretto 17 as Gradle JDK if have any compatibility issues

• Download NDK and ensure $ANDROID_NDK is visible in $PATH.

2.2. Deploying Model in android_deploy app

#

After converting the model to Relay (TVM), the model file must be placed in the assets/ folder in github . Then need to fix some gradle and compatibility . Also Need to setup the correct ndk-build version in the gradle files. The repository already have the download-models.gradle which will be downloaded on first launch. If you need to use the custom model, the download-models.gradle import must be disabled in the build.gradle file.

Note that whether you use the android_deploy from .zip or from the repository, it is missing a graph_executor_factory.cc in the jni/tvm_runtime.h, please add the line in.

Then running the task build from gradle will compile and build the correct library for TVM.

Common Pitfalls & Fixes

#