Building a Node.js API for Code Compilation using tmp and child_process, with Containerization
Virtual Compilers implement Code Compilation APIs to Compile and Execute the submitted Program. In this project let's understand the process of building a Code Compilation API that compiles programs in C++, C, Java, C#, Javascript and Python. We will also go Dockerize the application to make this more scalable.
Prerequisites
Before we begin, make sure you are familiar with Node.js . Additionally, you'll need Docker if you plan to deploy the application using containers. We need to install npm packages.
npm install express tmp cors
Setting up the server
We need to create a server.js file in the root directory of our project and set up the routes and run the server.
const path = require('path');
const express = require('express');
const app = express();
const port = 5000;
const cors = require('cors');
app.use(cors());
app.use(express.json());
//Setting route for the Compilation Route
app.use('/api/v1/compile', require('./routes/compileRoute'));
app.get('/',(req,res) => res.json({msg:"Server is running"}))
// Server runs at the port 5000
app.listen(port, () => {
console.log('Server running at port '+port);
});
Creating the Router Files
To avoid clustering of all routes in our index.js, we create route files in Express.js. It will help us to separate routes by creating separate files for each. So, let's write the compileRoute.js file in the routes folder.
const express = require('express')
const {compile} = require('../controllers/compileCode')
const router = express.Router()
//compileCode conroller handles POST request to /api/v1/compile
router.route('/').post(compile);
module.exports = router
Controller for Code Execution
Let’s create a folder inside the project folder, let’s call it controllers, inside this folder, create the compileCode.js file.
This file will be exporting the compile(req, res) function to handle the Client's POST Requests with code, language and input values in the Request body. The API responds with output and runtime values in JSON.
In the compile function, a temporary file is created using the tmp library. After the file is created, it is written with the Code received from the Client side. The suffix is appended to the temporary file name to include the extension like '.java' for Java and '.cpp' for C++.
After the file is written, the next challenge comes in executing the code. Execution of the languages also varies, for example, C, C++, and C# are compiled, Python and Javascript are interpreted, while Java is both compiled and interpreted :(
Usually, Node.js allows single-threaded, non-blocking performance but running a single thread in a CPU cannot handle increasing workload hence the child_process module can be used to spawn child processes. We will use the child_process.exec() function to spawn a shell first and then run a command within that shell. The stdout of the child process is appended to the result and is sent as a JSON response after execution.
Special Case of Java
Main.java because it strictly implements OOPs. The user should always define the class as public class Main to avoid ambiguity in file and class names.var tmp = require("tmp");
var fs = require("fs");
const { exec } = require("child_process");
async function compile(req, res) {
const { code, language, input } = req.body;
let result = "";
const extensions = {
java: ".java",
cpp: ".cpp",
c: ".c",
cs: ".cs",
python: ".py",
javascript: ".js",
};
tmp.file(
{ prefix: "project", postfix: extensions[language] },
async function (err, path, fd, cleanupCallback) {
if (err) throw err;
console.log("File: ", path);
console.log("Filedescriptor: ", fd);
if (language === "java") {
fs.rename(path, __dirname + "/Main.java", ()=>{
console.log("renamed");
});
fs.writeFile("Main.java", code, ()=>{
console.log("written file");
});
} else fs.writeFile(path, code, ()=>{
console.log("written file");
});
//writing the file with the code submmitted
const commands = {
java: `javac Main.java && java Main`,
cpp: `g++ ${path} -o outputCPP && ./outputCPP`,
c: `gcc ${path} -o outputC && ./outputC`,
cs: `mcs ${path} && mono ${path.slice(0, path.length - 3)}.exe`,
python: `python ${path}`,
javascript: `node ${path}`,
};
const initialTime = Date.now();
const child = exec(
commands[language],
{ stdio: "pipe" },
(error, stdout, stderr) => {
if (error) {
res.json({
output: error.message,
runtime: Date.now() - initialTime,
});
return;
}
res.json({output:stdout, runtime:Date.now() - initialTime });
},
);
// write the input to the stream
child.stdin.write(input);
//Main.class needs to be removed
if (language === "java")
fs.rm("Main.class", () => console.log("deleted class java"));
//end the input stream
child.stdin.end();
},
);
}
module.exports = {
compile,
};
Deployment
The process of deploying this application on a remote server can be challenging as we need the compilers and interpreters of languages preinstalled on our server.
To make our application scalable to remote servers we containerize it using Docker with all the compilation dependencies installed.
Create a Dockerfile in the root directory to create an image of this app. The Docker image should be strictly mapped to the assigned port.
# Use the Node.js base image
FROM node:14
# Install Python, Java, C++, C, C# dependencies
RUN apt-get update && apt-get install -y \
python3 \
default-jdk \
default-jre \
g++ \
mono-complete
# Set the working directory
WORKDIR /app
# Copy your Node.js application files into the container
COPY package.json package-lock.json ./
RUN npm install
# Copy the rest of your application files
COPY . .
# Expose the port your Node.js server will listen on
EXPOSE 5000
# Start your Node.js server
CMD ["node", "server.js"]
To build and run the Docker container locally, follow these steps:
Build the Docker image
docker build -t virtual-compiler .Run the Docker container
docker run -p 5000:5000 virtual-compiler
Thanks for reading so far and following the development of a Virtual Compiler :)