Yupeng Ma
← Back to projects

Case study · iFLYTEK Future · 2024-08 — 2024-11

Voice Cloning Backend

Production APIs and real-time streaming for an AI voice feature.

Role · Backend engineer for the Voice Double feature

  • Python
  • FastAPI
  • WebSocket
  • CozyVoice
  • Alibaba Cloud OSS
  • ELK
  • Nginx
A product client reaches a FastAPI control boundary, opens a streaming session, and coordinates with an AI model. Operational logs branch from the backend boundary.

  1. 01

    Overview

    Context
    A new headset feature needed to turn a user voiceprint into real-time synthesized speech.
    My contribution
    Designed and implemented the backend integration and its operational boundaries.
    Technology
    Python, FastAPI, WebSocket, CozyVoice, Alibaba Cloud OSS, ELK, and Nginx.
    Result
    Delivered the Voice Double backend for the 2024 Singles' Day product launch.

  2. 02

    The Problem

    The feature had to coordinate lifecycle requests, live synthesis, model serving, durable voiceprint storage, observability, deployment, and operations within one delivery window.

  3. 03

    My Role

    Backend engineer for the Voice Double feature

    • Designed and implemented HTTP lifecycle APIs and WebSocket streaming interfaces with FastAPI.
    • Integrated CozyVoice model serving and Alibaba Cloud OSS voiceprint storage.
    • Added the Nginx service boundary and ELK logging needed to operate and debug the backend in production.

  4. 04

    Architecture

    A focused map of the components and responsibility boundaries that matter to this project.

    Architecture

    Voice synthesis service boundaries

    1. 01Product client

      Lifecycle requests and streaming session

    2. 02Nginx boundary

      Reverse proxy and service entry

    3. 03Backend API

      FastAPI lifecycle operations

    4. 04Streaming

      WebSocket synthesis session

    5. 05Model service

      CozyVoice synthesis boundary

    6. 06Object storage

      Durable voiceprint objects

    7. 07Observability

      Centralized ELK logs

    • Product clientNginx boundaryservice traffic
    • Nginx boundaryBackend APIHTTP
    • Nginx boundaryStreamingWebSocket
    • Backend APIObject storagevoiceprints
    • StreamingModel servicesynthesis
    • Backend APIObservabilitylogs
    • StreamingObservabilitystream events
    The product client enters through a simplified Nginx boundary. FastAPI separates lifecycle APIs from the WebSocket streaming session, coordinates CozyVoice model serving and voiceprint storage, and emits operational events to centralized logging.

  5. 05

    How It Works

    The primary sequence that moves work or data through the system.

    Flow

    Lifecycle and streaming request flow

    1. 01Request

      HTTP operation or WebSocket session

    2. 02Validation

      Check request and session inputs

    3. 03Route

      Separate lifecycle from streaming work

    4. 04Lifecycle

      Coordinate voiceprint storage

    5. 05Streaming

      Coordinate live model synthesis

    6. 06Response

      Return operation or streamed output

    7. 07Logging

      Record service and stream context

    • RequestValidationreceive
    • ValidationRouteaccept
    • RouteLifecycleHTTP
    • RouteStreamingWebSocket
    • LifecycleResponsecomplete
    • StreamingResponsestream
    • ValidationLoggingrequest context
    • LifecycleLoggingoperation event
    • StreamingLoggingstream event
    A request enters the FastAPI boundary and is validated before routing. HTTP lifecycle work coordinates voiceprint storage and returns a bounded response; WebSocket work opens a streaming session that coordinates synthesis with the model service. Both paths emit operational context to centralized logging.

  6. 06

    Key Technical Decisions

    Problem
    Lifecycle operations and real-time synthesis had different interaction patterns.
    Choice
    Use HTTP for lifecycle operations and WebSocket for streaming synthesis.
    Why
    Each transport matched the duration and response behavior of the work it carried, keeping the public API easier to reason about.
    Trade-off
    The backend had to operate and observe two interface styles instead of a single transport.
    Problem
    Model inference, object storage, proxying, and logs could become tightly coupled inside one feature implementation.
    Choice
    Treat model serving, voiceprint storage, reverse proxying, and centralized logging as explicit integration boundaries.
    Why
    Clear boundaries made the production responsibilities and failure locations easier to understand and debug.
    Trade-off
    More boundaries introduced additional configuration and cross-service failure modes to manage.

  7. 07

    Challenges

    • Coordinating stateful voiceprint lifecycle operations with a live synthesis stream.
    • Making model integration observable enough for production debugging without exposing user data.

  8. 08

    Outcome

    • Delivered the backend for the Voice Double feature for the 2024 Singles' Day product launch.
    • Established explicit interfaces for model serving, storage, streaming, proxying, and centralized logs.

  9. 09

    What I Learned

    Learned
    Production AI features are integration systems, not just model calls; interface and operational boundaries matter as much as inference.
    Reflection
    Separating lifecycle control, real-time streaming, model serving, storage, and observability makes an AI feature easier to operate and explain.

This was an employer-owned production system. Architecture and implementation details shown here are simplified to protect confidential information.