@pexip/media

Enumerations

• DeniedDevices
• UserMediaStatus

Interfaces

• AudioDetectionSignals
• AudioSignalDetectionOptions
• DenoiseParams
• ExtendedMediaTrackSettings
• Media
• MediaAttributes
• MediaChangesSignals
• MediaController
• MediaInit
• MediaOptions
• MediaProps
• MediaSettings
• MediaTrack
• MediaTrackInit
• PreviewControllerProps
• PreviewEventHandler
• PreviewStreamController
• PreviewStreamParams
• StreamTrackSignals
• VoiceActivityDetectionOptions

Type Aliases

AudioContentHint

Ƭ AudioContentHint: typeof AUDIO_CONTENT_HINTS[keyof typeof AUDIO_CONTENT_HINTS]

---

CreatePreviewStreamController

Ƭ CreatePreviewStreamController: typeof createPreviewStreamController

---

ExtendedMediaTrackSettingsKey

Ƭ ExtendedMediaTrackSettingsKey: keyof ExtendedMediaTrackSettings

---

GeneratedMediaSignals

Ƭ GeneratedMediaSignals<K>: Pick<Required<MediaSignals>, K>

Type parameters

Name	Type
K	extends keyof MediaSignals

---

GetDisplayMedia

Ƭ GetDisplayMedia: ReturnType<typeof createGetDisplayMedia>

---

GetUserMediaProcess

Ƭ GetUserMediaProcess: (request: { constraints: MediaDeviceRequest ; currentMedia: Media | undefined ; originalConstraints: MediaDeviceRequest ; permission: { audio: PermissionState ; video: PermissionState } }) => Promise<Media>

Type declaration

▸ (request): Promise<Media>

Parameters

Name	Type
request	Object
request.constraints	MediaDeviceRequest
request.currentMedia	Media | undefined
request.originalConstraints	MediaDeviceRequest
request.permission	Object
request.permission.audio	PermissionState
request.permission.video	PermissionState

Returns

Promise<Media>

---

MediaSignals

Ƭ MediaSignals: MediaSignalsRequired & MediaSignalsOptional

---

MediaSignalsOptional

Ƭ MediaSignalsOptional: Omit<Partial<MediaChangesSignals>, "onMediaChanged"> & Partial<StreamTrackSignals> & Partial<StreamTrackSignalFinals>

---

MediaSignalsRequired

Ƭ MediaSignalsRequired: Pick<MediaChangesSignals, "onMediaChanged"> & AudioDetectionSignals

---

PreviewInput

Ƭ PreviewInput: MediaDeviceInfoLike | undefined

---

Process

Ƭ Process<T>: (a: T) => Promise<MediaTrack>

Type parameters

Name
T

Type declaration

▸ (a): Promise<MediaTrack>

Parameters

Name	Type
a	T

Returns

Promise<MediaTrack>

---

Segmenters

Ƭ Segmenters: { [Property in SegmentationModel]: Segmenter }

---

StreamTrackSignalFinals

Ƭ StreamTrackSignalFinals: { [key in keyof StreamTrackSignals as `${key}Final`]: Signal<MediaStreamTrack> }

Track state signals that used for the final track that the media pipeline produced

---

TrackProcessor

Ƭ TrackProcessor: Process<MediaTrack>

---

Unsubscribe

Ƭ Unsubscribe: () => void

Type declaration

▸ (): void

Returns

void

---

UserMediaValidator

Ƭ UserMediaValidator: (status: UserMediaStatus) => boolean

Type declaration

▸ (status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

VideoContentHint

Ƭ VideoContentHint: typeof VIDEO_CONTENT_HINTS[keyof typeof VIDEO_CONTENT_HINTS]

---

VideoRenderParams

Ƭ VideoRenderParams: Omit<RendererOptions, "effects"> & { frameRate: number ; height: number ; pan?: boolean ; tilt?: boolean ; videoSegmentation: RenderEffects ; width: number ; zoom?: boolean }

---

VideoStreamTrackProcessorAPIs

Ƭ VideoStreamTrackProcessorAPIs: "stream" | "canvas"

Use which processor API to process the stream track stream - Use MediaStreamTrackProcessor, when available canvas - Use Canvas

Variables

AUDIO_CONTENT_HINTS

• Const AUDIO_CONTENT_HINTS: Object

Audio content hints are only applicable when the MediaStreamTrack contains an audio track

https://www.w3.org/TR/mst-content-hint/#audio-content-hints

Type declaration

Name	Type	Description
Music	"music"	The track should be treated as if it contains music data. Generally this might imply tuning or turning off audio-processing components that are used to process speech data to prevent the audio from being distorted.
NoHint	""	No hint has been provided, the implementation should make its best-informed guess on how to handle contained audio data. This may be inferred from how the track was opened or by doing content analysis
Speech	"speech"	The track should be treated as if it contains speech data. Consuming this signal it may be appropriate to apply noise suppression or boost intelligibility of the incoming signal.
SpeechRecognition	"speech-recognition"	The track should be treated as if it contains data for the purpose of speech recognition by a machine. Consuming this signal it may be appropriate to boost intelligibility of the incoming signal for transcription and turn off audio-processing components that are used for human consumption.

---

REQUIRED_SIGNAL_KEYS

• Const REQUIRED_SIGNAL_KEYS: readonly ["onMediaChanged", "onVAD", "onSilentDetected"]

---

VIDEO_CONTENT_HINTS

• Const VIDEO_CONTENT_HINTS: Object

Video content hints are only applicable when the MediaStreamTrack contains a video track.

https://www.w3.org/TR/mst-content-hint/#video-content-hints

Type declaration

Name	Type	Description
Detail	"detail"	The track should be treated as if video details are extra important. This is generally applicable to presentations or web pages with text content, painting or line art. This setting would normally optimize for detail in the resulting individual frames rather than smooth playback. Artefacts from quantization or downscaling that make small text or line art unintelligible should be avoided.
Motion	"motion"	The track should be treated as if it contains video where motion is important. This is normally webcam video, movies or video games. Quantization artefacts and downscaling are acceptable in order to preserve motion as well as possible while still retaining target bitrates. During low bitrates when compromises have to be made, more effort is spent on preserving frame rate than edge quality and details.
NoHint	""	No hint has been provided, the implementation should make its best-informed guess on how contained video content should be treated. This can for example be inferred from how the track was opened or by doing content analysis.
Text	"text"	The track should be treated as if video details are extra important, and that significant sharp edges and areas of consistent color can occur frequently. This is generally applicable to presentations or web pages with text content. This setting would normally optimize for detail in the resulting individual frames rather than smooth playback, and may take advantage of encoder tools that optimize for text rendering. Artefacts from quantization or downscaling that make small text or line art unintelligible should be avoided.

---

internalSignals

• Const internalSignals: Object

Internal signals

Type declaration

Name	Type
onDevicesChanged	Signal<IndexedDevices>

Functions

applyContentHint

▸ applyContentHint<T>(hint?): (track: MediaStreamTrack) => void

Apply the content hint to the track

Type parameters

Name	Type
T	extends "" | "speech" | "speech-recognition" | "music" | "motion" | "detail" | "text"

Parameters

Name	Type	Description
hint?	T	Content hint

Returns

fn

▸ (track): void

Parameters

Name	Type
track	MediaStreamTrack

Returns

void

---

areBothGranted

▸ areBothGranted(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

createAudioMixingProcess

▸ createAudioMixingProcess(getCurrentMedia, label?): TrackProcessor

Create a Audio Mixing Processor and will own the stream passed-in

Parameters

Name	Type	Default value
getCurrentMedia	() => undefined | MediaStream	undefined
label	"AudioMixingProcessor"	PROCESSOR_LABELS.AudioMixingProcessor

Returns

TrackProcessor

---

createAudioStreamProcess

▸ createAudioStreamProcess(«destructured»): TrackProcessor

Create a Audio Stream Processor and will own the stream passed-in

Parameters

Name	Type
«destructured»	AudioProcessOptions & ThrottleOptions

Returns

TrackProcessor

---

createGetDisplayMedia

▸ createGetDisplayMedia(getDefaultConstraints, getDisplayMedia?): (constraints?: DisplayMediaOptions) => Promise<undefined | MediaStream>

Parameters

Name	Type
getDefaultConstraints	() => DisplayMediaOptions
getDisplayMedia	(options?: DisplayMediaStreamOptions) => Promise<MediaStream>

Returns

fn

▸ (constraints?): Promise<undefined | MediaStream>

Parameters

Name	Type
constraints?	DisplayMediaOptions

Returns

Promise<undefined | MediaStream>

---

createMedia

▸ createMedia(options): MediaController

Create an object to interact with the media scream, which is usually used for our main stream.

See

MediaOptions

Parameters

Name	Type
options	MediaOptions

Returns

MediaController

---

createMediaSignal

▸ createMediaSignal<T>(name, crucial?, variant?): Signal<T>

Create a general signal with consistent scoped name

Default Value

true

See

Signal

Default Value

'generic'

Type parameters

Name	Type
T	undefined

Parameters

Name	Type	Default value	Description
name	string	undefined	Signal name
crucial	boolean	true	Signify if the signal is unmissable.
variant	SignalVariant	'generic'	The variant of the signal

Returns

Signal<T>

---

createMediaSignals

▸ createMediaSignals<K>(more, scope?): Pick<Required<MediaSignals>, SignalKeys>

Create and return all required and optional (if specified with more), signals for media to work

See

• MediaSignalsOptional
• Signal

The following signals created by default

• 'onMediaChanged',
• 'onVAD',
• REQUIRED_SIGNAL_KEYS

Type parameters

Name	Type
K	extends "onAddTrack" | "onAudioMuteStateChanged" | "onTrackReleased" | "onDevicesChanged" | "onRemoveTrack" | "onStatusChanged" | "onVideoMuteStateChanged" | "onUpdatingMedia" | keyof StreamTrackSignals | "onStreamTrackMutedFinal" | "onStreamTrackUnmutedFinal" | "onStreamTrackEndedFinal" | "onStreamTrackEnabledFinal"

Parameters

Name	Type	Default value	Description
more	K[]	undefined	Keys from MediaSignalsOptional,
scope	string	''	any scope prefix for the generated signal name,

Returns

Pick<Required<MediaSignals>, SignalKeys>

---

createPreviewStreamController

▸ createPreviewStreamController(«destructured»): PreviewStreamController

Parameters

Name	Type
«destructured»	PreviewStreamParams

Returns

PreviewStreamController

---

createVideoStreamProcess

▸ createVideoStreamProcess(«destructured»): TrackProcessor

Parameters

Name	Type
«destructured»	VideoStreamProcessOptions

Returns

TrackProcessor

---

deriveInitialPermissionStatus

▸ deriveInitialPermissionStatus(prevStatus, getPermissionState?): Promise<UserMediaStatus>

Parameters

Name	Type	Default value
prevStatus	UserMediaStatus	undefined
getPermissionState	(anyActiveAudioTrack?: boolean, anyActiveVideoTrack?: boolean, getDevices?: () => Promise<MediaDeviceInfo[]>) => Promise<InputDevicePermission>	getInputDevicePermissionState

Returns

Promise<UserMediaStatus>

---

getPermissionStatus

▸ getPermissionStatus(«destructured»): Initial | InitialPermissionsGranted | InitialPermissionsNotGranted | InitialPermissionsVideoInputDenied | InitialPermissionsAudioInputDenied | InitialPermissionsVideoInputGranted | InitialPermissionsAudioInputGranted | InitialPermissionsGrantedVideoInputDenied | InitialPermissionsGrantedAudioInputDenied | PermissionsRejected

Parameters

Name	Type
«destructured»	Object
› audio	PermissionState
› video	PermissionState

Returns

Initial | InitialPermissionsGranted | InitialPermissionsNotGranted | InitialPermissionsVideoInputDenied | InitialPermissionsAudioInputDenied | InitialPermissionsVideoInputGranted | InitialPermissionsAudioInputGranted | InitialPermissionsGrantedVideoInputDenied | InitialPermissionsGrantedAudioInputDenied | PermissionsRejected

---

hasNoAudioDevices

▸ hasNoAudioDevices(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

hasNoDevice

▸ hasNoDevice(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

hasNoVideoDevices

▸ hasNoVideoDevices(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isAudioDeviceInUse

▸ isAudioDeviceInUse(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isDeviceInUse

▸ isDeviceInUse(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isDevicesInUse

▸ isDevicesInUse(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isFallback

▸ isFallback(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isFallbackAudio

▸ isFallbackAudio(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isFallbackVideo

▸ isFallbackVideo(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isGranted

▸ isGranted(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isGrantedAudio

▸ isGrantedAudio(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isGrantedOnlyAudio

▸ isGrantedOnlyAudio(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isGrantedOnlyAudioNoVideoDevices

▸ isGrantedOnlyAudioNoVideoDevices(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isGrantedOnlyVideo

▸ isGrantedOnlyVideo(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isGrantedOnlyVideoNoAudioDevices

▸ isGrantedOnlyVideoNoAudioDevices(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isGrantedVideo

▸ isGrantedVideo(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isInitial

▸ isInitial(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isInitialPermissions

▸ isInitialPermissions(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isInitialPermissionsGranted

▸ isInitialPermissionsGranted(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isInitialPermissionsNotGranted

▸ isInitialPermissionsNotGranted(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isOnlyAudioError

▸ isOnlyAudioError(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isOnlyVideoError

▸ isOnlyVideoError(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isOverConstrained

▸ isOverConstrained(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isPromptAudio

▸ isPromptAudio(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isPromptVideo

▸ isPromptVideo(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isRejected

▸ isRejected(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isRejectedOnlyAudio

▸ isRejectedOnlyAudio(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isRejectedOnlyVideo

▸ isRejectedOnlyVideo(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isUnknownError

▸ isUnknownError(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

isVideoDeviceInUse

▸ isVideoDeviceInUse(status): boolean

Parameters

Name	Type
status	UserMediaStatus

Returns

boolean

---

setLogger

▸ setLogger(newLogger): void

Parameters

Name	Type
newLogger	Logger

Returns

void

---

toDeniedDevices

▸ toDeniedDevices(status?): undefined | DeniedDevices

Parameters

Name	Type
status?	UserMediaStatus

Returns

undefined | DeniedDevices