Andy Zeng, Chief Technology Officer at BrewAI, is an experienced AI professional with expertise in building AI platforms with deep learning and LLMs. Zeng is also the co-founder of Deep Learning Sydney Meetup. He holds a Doctor of Philosophy – PhD, Machine Learning from the Institute of Computing Technology, Chinese Academy of Sciences.
As the co-founder and CTO of BrewAI, his strong background in machine learning, artificial intelligence, and software development enables the company to shape its technological direction and create cutting-edge AI solutions. The company eases AI implementation for businesses and reduces the time and cost required to produce AI models and services, thus making them ready to use for end users.
Zeng’s AI Journey
Andy Zeng recalls several experiences that inspired him to kick-start his technology journey and shaped his path to becoming a CTO. 20 years ago, he was a Ph.D. student doing research in Machine Learning. He discovered that AI is mighty and would definitely transform the world in the near future.
Intrigued by AI’s future and abilities, he decided to be a part of the league. Post completion of his Ph.D., he then worked as a postdoctoral research fellow at the University of Technology Sydney (UTS).
At the university, he worked on an at-risk student discovery project that aimed to predict students who had academic difficulties. He built a system called i-Educator that helped the Student Service Unit do early interventions for high-risk students before they drop out of school. The project was so successful and impactful that he got a chance to run a workshop with UTS’ Vice Chancellor in an International Educational Conference called SoLAR Flare. This success further strengthened his dream of becoming a tech leader and making a difference in the world with his technical strength.
One more experience worth mentioning was his career at Tabcorp, the largest gaming firm in Australia, where he conducted a project to predict gamblers who were addicted to gambling and may end up committing suicide if they incur heavy losses. With the support of the AI project, Tabcorp initiated a hotline to help them, which saved hundreds of high-risk gamblers’ lives.
With motivations from all those experiences, he kept working harder towards his goals. He then became an associate director at KPMG, a director of AI at JollyChic, and finally the CTO of BrewAI.
Driving Innovation at Scale
Zeng shed light on the several steps he followed to shape an innovative tech team at BrewAI. The foremost step was to hire the right candidates. He is always keen to hire candidates who are pioneers in the industry, who are always keen to explore the latest AI theories and techniques, and who are very full-stack, not only on frontend and backend, but also on AI, cloud, infra, and hardware.
The second step is to build the right culture for the company. Andy states he has intentionally maintained the tech team as a flat organization and made himself easily reachable for each member. Meanwhile, he is open to any proposals on new technologies for trial, on new directions for testing. To create a close-knit team, the heads arrange pairing sessions for junior developers. They have brown bag sessions for knowledge sharing and regular Friday lunches for team building. In this way, they have built an active engineering team that is proactive, forward-thinking, and always doing cool things.
The third step is to align everyone to the same vision and make it achievable. The leadership team sets a vision that will excite everyone. At the same time, they divide the vision into a list of milestones. Each milestone is then broken down into a sequence of achievable tasks, which will be carefully evaluated and estimated. By following these guidelines, vision becomes a pathway that is approachable in a reasonable timeframe.
Redefining Leadership
As per Andy, leadership means influence, impact, and the capability of leading a team to success. He got leadership roles very early in high school and at university. However, his experience of co-founding Deep Learning Sydney Meetup highlights a redefining leadership in his career line.
By 2015, deep learning methodologies like CNN and RNN became very successful with rapid development in large datasets (like ImageNet) and high-performance GPUs. However, the industry still focused on terminologies like “Big Data” and “Data Science”. This meant that the industry’s focuses were still on building the right data pipelines and leveraging traditional machine learning methods for industrial applications. He believed that it was time for deep learning to influence the industry. He then co-founded Deep Learning Sydney with a friend who was a professor at UTS.
While establishing the meetup, he wanted it to be a bridge between academia and industry so that the latest developments in AI can be adopted by industry easily. He also wanted the meetup to be a place where the best AI demands can meet the best AI talents. The meetup group grew substantially, and now they have around 5000 members, making it the biggest community on Deep Learning and AI in Australia.
One of the worth mentioning events is Deep Learning Sydney sponsored Facebook’s (now rebranded as Meta) hiring event in Australia in 2016. The team helped Facebook find a lot of qualified candidates from their group. Even Andy was offered the opportunity to join Facebook. However, he didn’t take that opportunity but took an associate director role at KPMG. The reason was to pursue a leadership role and influence industrial clients directly towards growth and fuel AI with futuristic possibilities.
Balancing Leadership and Strategic Vision
As per Andy Zeng, technical leadership and strategic foresight go hand-in-hand. For him, understanding the tech trend is more important than drilling into technical details for a CTO role. He has hired tech gurus who really understand the nitty-gritty of different techniques while he focuses on the impactful tech trends. He believes that having a good understanding of the tech trends helps him work well on the strategic foresight for the company.
Andy relies on the tech gurus he has hired for making detailed decisions on tech solutions. On the other hand, he looks at strategic planning, focusing on how to make their products pioneers in the market, and following the steps to achieve that target.
Mission Focus vs Experimentation
BrewAI has a mission to bring transformative AI solutions to the industry. Commenting on the company’s mission, Andy stated that the mission is always of the highest priority. Everything they do is to fulfil BrewAI’s mission. However, this does not mean that the team doesn’t have time for creative thinking and experimentation. In fact, BrewAI’s mission embraces experimentation in nature. Providing transformative solutions, out-of-the-box ideas, and going beyond boundaries are essential.
For example, hallucination is a well-known issue for LLMs, which is also very harmful for building AI products. Instead of waiting for academia to resolve it at the theoretical level, BrewAI urgently needed to conquer it to make LLM’s output trustworthy. To achieve this, they encouraged the team to think out of the box and evaluate their thoughts through various experiments. Luckily, they got a solution from the engineering perspective, i.e., providing guardrails to make sure LLMs’ output has references that they can trust.
Developing Private LLMs
Furthermore, explaining the challenges while developing private LLMS, Andy stated that building private LLM solutions is quite different from building solutions with public LLM APIs (e.g., OpenAI APIs). When their customers choose a private LLM solution, they usually treat data privacy as an essential consideration in their organization. The challenges are usually from these aspects, i.e., the LLMs should be air-gapped to guarantee there is zero data leakage. The solution should meet regulatory standards so that the industry can use it comfortably, and the platform should be easy to use, with minimal technical maintenance.
BrewAI has pushed to the market a concept called “Brew-in-a-Box”. Their offer is a one-piece solution. It is a combination of hardware and software as they customize the right hardware (including GPUs, CPUs, memory, and hard disks) for their client to form a one-U or two-U server. Their LLM solution will be installed on this server, well-encrypted, meeting all regulatory standards. The “Brew-in-a-Box” will then be put in an air-gapped server room, either owned by the client or from a third-party provider. They also provide offline packages for clients to do system upgrades. Lastly, they provide a high SLA standard to their clients to make sure they get looked after 24/7.
Andy also shared his insight on one of BrewAI’s private LLM projects that left a transformative mark on the clients. It was a project they did with a major legal firm in Australia. Their pain point is that they need to deal with lots of legal materials daily. Most of the time, junior lawyers’ tasks are to find answers to a specific set of questions among different legal materials. This is a typical use case for their LLM solution. Based on one of their core products called BrewChat, they created a legal version, BrewLegal, which mainly focuses on legal pain points. With this product, they can easily chat on a specific legal matter and get their queries solved, with references highlighted. They can also run batch jobs to get reports that align with the client’s standards in documented form. The lawyers can work comfortably with well-designed access control according to their organization’s needs.
Also, the system complies with the Australian regulations on using LLMs for the legal industry. With this product, their junior lawyers’ efficiency has been improved 10x, leaving them more spare time to work on important matters. They believe this is a game changer for them and for the industry.
Work Life Balance
Speaking about work-life integration, Andy claims work-life balance is always an issue for entrepreneurs. There are a lot of things to deal with, and the company evolves fast. However, for him, looking after his family is as important as bringing a company to success. He explains that some scenarios during family time often help one with an idea to solve the complex company problems.
Also, hiring the team carefully and trusting those aids in the division of the workload. More importantly, as the CTO, he should use the time wisely, for example, putting more effort into the vision and defining the pathway to achieve it. A well-defined pathway will save tons of work, and at the same time, reduce the time to success for the company.
Evolving with Emerging Trends
Since AI and machine learning evolve very fast, Andy keeps himself up-to-date by following several steps. To begin with, as the co-founder and organizer of Deep Learning Sydney, they organize meetups regularly. It becomes a channel to get connected to the latest updates in AI. He has reserved 20% of his time to learn new things, like reading papers, articles, and learning from other companies’ solutions. He is always hands-on with emerging trends, especially in AI. No matter what’s new, he will always try to see if it’s a value addition for his company.
Mentoring and Leading with Purpose
Andy firmly believes that mentorship is important if one wants to retain their staff in the long term. As a head, he focuses on two aspects, i.e., how to help the staff grow with the company, and how to find the right future roles for them.
To figure these problems out, the first task is to listen to the staff. Andy has a close relationship with the team; they speak daily on several occasions. He gets to understand their motivations and what the idea of the future is for them.
For him, having a successful company is not just about achieving the company goals; it is also about helping staff to grow into what they’ve dreamed of.
Meaningful Achievements
Shedding light on his most meaningful achievement, Andy states that it’s yet to be achieved in the future. However, the things that he is proud of are all about his positive contribution to society. For example, he is proud of the i-Educator system he created at the university, which helped hundreds of students facing difficulties with academics. His high-risk gambler prevention project at Tabcorp, which has saved lots of gamblers’ lives, is also one of his greatest achievements. He is also proud of cofounding the Deep Learning Sydney Meetup, which has been a bridge for academia and industry for almost ten years. During the past ten years, the meetup team has provided hundreds of insightful talks to their audience, and they have helped companies with hiring and AI experts find their ideal opportunities.
Now at BrewAI, they are building transformative solutions for different sectors. The team has achieved a lot; however, the most exciting achievement is yet to come.
Advice to Aspiring CTOs
While advising aspiring CTOs wanting to create a real difference in the AI world, Andy expresses that AI is a powerful tool and that one should always use it for a good purpose. The future generation should always think of ways they can positively contribute to the world with their tech expertise. Focus on how they can help their customers in achieving the desired goal and how they can aid the sector with cost-reduction strategies.
Andy Zeng strongly believes AI is for everyone and every organization. AI should be secure, ethically aware, and affordable. Therefore, they have put a lot of effort into on-prem AI solutions. They strive to keep their platform and service costs reasonable and affordable for all users, at the same time ensuring data privacy and compliance requirements are met. He believes their efforts can encourage a responsible AI future, which puts data privacy and ethical requirements in an essential position. Through this way, they can foresee a future where every organization is able to have its own LLMs to help it evolve its business.
