巴延興

深圳開鴻數(shù)字產(chǎn)業(yè)發(fā)展有限公司

資深OS框架開發(fā)工程師

以下內(nèi)容來自嘉賓分享，不代表開放原子開源基金會(huì)觀點(diǎn)

一、簡介

媒體子系統(tǒng)為開發(fā)者提供一套接口，方便開發(fā)者使用系統(tǒng)的媒體資源，主要包含音視頻開發(fā)、相機(jī)開發(fā)、流媒體開發(fā)等模塊。每個(gè)模塊都提供給上層應(yīng)用對(duì)應(yīng)的接口，本文會(huì)對(duì)音視頻開發(fā)中的音視頻播放框架做一個(gè)詳細(xì)的介紹。

二、目錄

foundation/multimedia/media_standard

├── frameworks                        #框架代碼
│ ├── js
│ │ ├── player
│ ├── native
│ │ ├── player                    #native實(shí)現(xiàn)
│ └── videodisplaymanager         #顯示管理
│     ├── include
│     └── src
├── interfaces
│ ├── inner_api                     #內(nèi)部接口
│ │ └── native
│ └── kits                          #外部JS接口
├── sa_profile                        #服務(wù)配置文件
└── services
  ├── engine                        #engine代碼
  │ └── gstreamer
  ├── etc                           #服務(wù)配置文件
  ├── include                       #頭文件
  └── services
    ├── sa_media                  #media服務(wù)
    │ ├── client                #media客戶端
    │ ├── ipc                   #media ipc調(diào)用
    │ └── server                #media服務(wù)端
    ├── factory                   #engine工廠
    └── player                    #player服務(wù)
     ├── client                 #player客戶端
      ├── ipc                    #player ipc調(diào)用
└──server#player服務(wù)端

三、播放的總體流程

?

四、Native接口使用

OpenHarmony系統(tǒng)中，音視頻播放通過N-API接口提供給上層JS調(diào)用，N-API相當(dāng)于是JS和Native之間的橋梁，在OpenHarmony源碼中，提供了C++直接調(diào)用的音視頻播放例子，在foundation/multimedia/player_framework/test/nativedemo/player目錄中。

void PlayerDemo::RunCase(const string &path)
{
    player_ = OHOS::CreatePlayer();
    if (player_ == nullptr) {
        cout << "player_ is null" << endl;
        return;
    }
    RegisterTable();
    std::shared_ptr cb = std::make_shared();
    cb->SetBufferingOut(SelectBufferingOut());


    int32_t ret = player_->SetPlayerCallback(cb);
    if (ret != 0) {
        cout << "SetPlayerCallback fail" << endl;
    }
    if (SelectSource(path) != 0) {
        cout << "SetSource fail" << endl;
        return;
    }
    sptr producerSurface = nullptr;
    producerSurface = GetVideoSurface();
    if (producerSurface != nullptr) {
        ret = player_->SetVideoSurface(producerSurface);
        if (ret != 0) {
            cout << "SetVideoSurface fail" << endl;
        }
    }
    SetVideoScaleType();
    if (SelectRendererMode() != 0) {
        cout << "set renderer info fail" << endl;
    }
    ret = player_->PrepareAsync();
    if (ret !=  0) {
        cout << "PrepareAsync fail" << endl;
        return;
    }
    cout << "Enter your step:" << endl;
    DoNext();
}

首先根據(jù)RunCase可以大致了解一下播放音視頻的主要流程，創(chuàng)建播放器，設(shè)置播放源，設(shè)置回調(diào)方法（包含播放過程中的多種狀態(tài)的回調(diào)），設(shè)置播放顯示的Surface，這些準(zhǔn)備工作做好之后，需要調(diào)用播放器的PrepareASync方法，這個(gè)方法完成后，播放狀態(tài)會(huì)變成Prepared狀態(tài)，這時(shí)就可以調(diào)用播放器的play接口，進(jìn)行音視頻的播放了。 RegisterTable()方法中，將字符串和對(duì)應(yīng)的方法映射到Map中，這樣后續(xù)的DoNext會(huì)根據(jù)輸入的命令，來決定播放器具體的操作。

void PlayerDemo::DoNext()
{
    std::string cmd;
    while (std::cin, cmd)) {
        auto iter = playerTable_.find(cmd);
        if (iter != playerTable_.end()) {
            auto func = iter->second;
            if (func() != 0) {
                cout << "Operation error" << endl;
            }
            if (cmd.find("stop") != std::npos && dataSrc_ != nullptr) {
                dataSrc_->Reset();
            }
            continue;
        } else if (cmd.find("quit") != std::npos || cmd == "q") {
            break;
        } else {
            DoCmd(cmd);
            continue;
        }
    }
}


void PlayerDemo::RegisterTable()
{
    (void)playerTable_.emplace("prepare", std::bind(&Player::Prepare, player_));
    (void)playerTable_.emplace("prepareasync", std::bind(&Player::PrepareAsync, player_));
    (void)playerTable_.emplace("", std::bind(&Player::Play, player_)); // ENTER -> play
    (void)playerTable_.emplace("play", std::bind(&Player::Play, player_));
    (void)playerTable_.emplace("pause", std::bind(&Player::Pause, player_));
    (void)playerTable_.emplace("stop", std::bind(&Player::Stop, player_));
    (void)playerTable_.emplace("reset", std::bind(&Player::Reset, player_));
    (void)playerTable_.emplace("release", std::bind(&Player::Release, player_));
    (void)playerTable_.emplace("isplaying", std::bind(&PlayerDemo::GetPlaying, this));
    (void)playerTable_.emplace("isloop", std::bind(&PlayerDemo::GetLooping, this));
    (void)playerTable_.emplace("speed", std::bind(&PlayerDemo::GetPlaybackSpeed, this));
}

以上的DoNext方法中核心的代碼是func()的調(diào)用，這個(gè)func就是之前注冊進(jìn)Map中字符串對(duì)應(yīng)的方法，在RegisterTable方法中將空字符串""和"play"對(duì)綁定為Player::Play方法，默認(rèn)不輸入命令參數(shù)時(shí)，是播放操作。

五、調(diào)用流程

? ?

左右滑動(dòng)查看更多

本段落主要針對(duì)媒體播放的框架層代碼進(jìn)行分析，所以在流程中涉及到了IPC調(diào)用相關(guān)的客戶端和服務(wù)端，代碼暫且分析到調(diào)用gstreamer引擎。 首先Sample通過PlayerFactory創(chuàng)建了一個(gè)播放器實(shí)例（PlayerImpl對(duì)象），創(chuàng)建過程中調(diào)用Init函數(shù)。

int32_t PlayerImpl::Init()
{
    playerService_ = MediaServiceFactory::GetInstance().CreatePlayerService();
    CHECK_AND_RETURN_RET_LOG(playerService_ != nullptr, MSERR_UNKNOWN, "failed to create player service");
    return MSERR_OK;
}

MediaServiceFactory::GetInstance()返回的是MediaClient對(duì)象，所以CreateplayerService函數(shù)實(shí)際上是調(diào)用了MediaClient對(duì)應(yīng)的方法。

std::shared_ptr MediaClient::CreatePlayerService()
{
    std::lock_guard lock(mutex_);
    if (!IsAlived()) {
        MEDIA_LOGE("media service does not exist.");
        return nullptr;
    }


    sptr object = mediaProxy_->GetSubSystemAbility(
        IStandardMediaService::MEDIA_PLAYER, listenerStub_->AsObject());
    CHECK_AND_RETURN_RET_LOG(object != nullptr, nullptr, "player proxy object is nullptr.");


    sptr playerProxy = iface_cast(object);
    CHECK_AND_RETURN_RET_LOG(playerProxy != nullptr, nullptr, "player proxy is nullptr.");


    std::shared_ptr player = PlayerClient::Create(playerProxy);
    CHECK_AND_RETURN_RET_LOG(player != nullptr, nullptr, "failed to create player client.");


    playerClientList_.push_back(player);
    return player;
}

這個(gè)方法中主要通過PlayerClient::Create(playerProxy)方法創(chuàng)建了PlayerClient實(shí)例，并且將該實(shí)例一層層向上傳，最終傳給了PlayerImpl的playerService_變量，后續(xù)對(duì)于播放器的操作，PlayerImpl都是通過調(diào)用PlayerClient實(shí)例實(shí)現(xiàn)的。

int32_t PlayerImpl::Play()
{
    CHECK_AND_RETURN_RET_LOG(playerService_ != nullptr, MSERR_INVALID_OPERATION, "player service does not exist..");
    MEDIA_LOGW("KPI-TRACE: PlayerImpl Play in");
    return playerService_->Play();
}


int32_t PlayerImpl::Prepare()
{
    CHECK_AND_RETURN_RET_LOG(playerService_ != nullptr, MSERR_INVALID_OPERATION, "player service does not exist..");
    MEDIA_LOGW("KPI-TRACE: PlayerImpl Prepare in");
    return playerService_->Prepare();
}


int32_t PlayerImpl::PrepareAsync()
{
    CHECK_AND_RETURN_RET_LOG(playerService_ != nullptr, MSERR_INVALID_OPERATION, "player service does not exist..");
    MEDIA_LOGW("KPI-TRACE: PlayerImpl PrepareAsync in");
    return playerService_->PrepareAsync();
}

對(duì)于PlayerImpl來說，playerService_指向的PlayerClient就是具體的實(shí)現(xiàn)，PlayerClient的實(shí)現(xiàn)是通過IPC的遠(yuǎn)程調(diào)用來實(shí)現(xiàn)的，具體地是通過IPC中的proxy端向遠(yuǎn)端服務(wù)發(fā)起遠(yuǎn)程調(diào)用請(qǐng)求。 我們以播放Play為例：

int32_t PlayerClient::Play()
{
    std::lock_guard lock(mutex_);
    CHECK_AND_RETURN_RET_LOG(playerProxy_ != nullptr, MSERR_NO_MEMORY, "player service does not exist..");
    return playerProxy_->Play();
}

int32_t PlayerServiceProxy::Play()
{
    MessageParcel data;
    MessageParcel reply;
    MessageOption option;


    if (!data.WriteInterfaceToken(PlayerServiceProxy::GetDescriptor())) {
        MEDIA_LOGE("Failed to write descriptor");
        return MSERR_UNKNOWN;
    }


    int error = Remote()->SendRequest(PLAY, data, reply, option);
    if (error != MSERR_OK) {
        MEDIA_LOGE("Play failed, error: %{public}d", error);
        return error;
    }
    return reply.ReadInt32();
}

proxy端發(fā)送調(diào)用請(qǐng)求后，對(duì)應(yīng)的Stub端會(huì)在PlayerServiceStub::OnRemoteRequest接收到請(qǐng)求，根據(jù)請(qǐng)求的參數(shù)進(jìn)行對(duì)應(yīng)的函數(shù)調(diào)用。播放操作對(duì)應(yīng)的調(diào)用Stub的Play方法。

int32_t PlayerServiceStub::Play()
{
    MediaTrace Trace("binder::Play");
    CHECK_AND_RETURN_RET_LOG(playerServer_ != nullptr, MSERR_NO_MEMORY, "player server is nullptr");
    return playerServer_->Play();
}

這里最終是通過playerServer_調(diào)用Play函數(shù)。playerServer_在Stub初始化的時(shí)候通過PlayerServer::Create()方式來獲取得到。也就是PlayerServer。

std::shared_ptr PlayerServer::Create()
{
    std::shared_ptr server = std::make_shared();
    CHECK_AND_RETURN_RET_LOG(server != nullptr, nullptr, "failed to new PlayerServer");


    (void)server->Init();
    return server;
}

最終我們的Play調(diào)用到了PlayerServer的Play()。在媒體播放的整個(gè)過程中會(huì)涉及到很多的狀態(tài)，所以在Play中進(jìn)行一些狀態(tài)的判讀后調(diào)用OnPlay方法。這個(gè)方法中發(fā)起了一個(gè)播放的任務(wù)。

int32_t PlayerServer::Play()
{
    std::lock_guard lock(mutex_);


    if (lastOpStatus_ == PLAYER_PREPARED || lastOpStatus_ == PLAYER_PLAYBACK_COMPLETE ||
        lastOpStatus_ == PLAYER_PAUSED) {
        return OnPlay();
    } else {
        MEDIA_LOGE("Can not Play, currentState is %{public}s", GetStatusDescription(lastOpStatus_).c_str());
        return MSERR_INVALID_OPERATION;
    }
}


int32_t PlayerServer::OnPlay()
{
    auto playingTask = std::make_shared>([this]() {
        MediaTrace::TraceBegin("PlayerServer::Play", FAKE_POINTER(this));
        auto currState = std::static_pointer_cast(GetCurrState());
        (void)currState->Play();
    });


    int ret = taskMgr_.LaunchTask(playingTask, PlayerServerTaskType::STATE_CHANGE);
    CHECK_AND_RETURN_RET_LOG(ret == MSERR_OK, ret, "Play failed");


    lastOpStatus_ = PLAYER_STARTED;
    return MSERR_OK;
}

在播放任務(wù)中調(diào)用了PlayerServer::Play()

int32_t PlayerServer::Play()
{
    return server_.HandlePlay();
}

在Play里面直接調(diào)用PlayerServer的HandlePlay方法，HandlePlay方法通過playerEngine_調(diào)用到了gstreamer引擎，gstreamer是最終播放的實(shí)現(xiàn)。

int32_t PlayerServer::HandlePlay()
{
    int32_t ret = playerEngine_->Play();
    CHECK_AND_RETURN_RET_LOG(ret == MSERR_OK, MSERR_INVALID_OPERATION, "Engine Play Failed!");


    return MSERR_OK;
}

六、總結(jié)

本文主要對(duì)OpenHarmony 3.2 Beta多媒體子系統(tǒng)的媒體播放進(jìn)行介紹，首先梳理了整體的播放流程，然后對(duì)播放的主要步驟進(jìn)行了詳細(xì)地分析。 媒體播放主要分為以下幾個(gè)層次：

（1）提供給應(yīng)用調(diào)用的Native接口，這個(gè)實(shí)際上通過OHOS::CreatePlayer()調(diào)用返回PlayerImpl實(shí)例。

（2）PlayerClient，這部分通過IPC的proxy調(diào)用，向遠(yuǎn)程服務(wù)發(fā)起調(diào)用請(qǐng)求。

（3）PlayerServer，這部分是播放服務(wù)的實(shí)現(xiàn)端，提供給Client端調(diào)用。

（4）Gstreamer，這部分是提供給PlayerServer調(diào)用，真正實(shí)現(xiàn)媒體播放的功能。

更多熱點(diǎn)文章閱讀

• 玩嗨OpenHarmony：基于OpenHarmony的智能助老服務(wù)機(jī)器人
• 玩嗨OpenHarmony：基于OpenHarmony的智慧農(nóng)業(yè)環(huán)境監(jiān)控系統(tǒng)
• 基于OpenHarmony的智慧牧場方案：生物姿態(tài)檢測篇
• 基于OpenHarmony的智慧牧場方案：生物心率檢測篇
• 使用ADS1115擴(kuò)展ROC-RK3568-PC開發(fā)板ADC功能
  

提示：本文由電子發(fā)燒友社區(qū)發(fā)布，轉(zhuǎn)載請(qǐng)注明以上來源。如需社區(qū)合作及入群交流，請(qǐng)?zhí)砑游⑿臙EFans0806，或者發(fā)郵箱liuyong@huaqiu.com。