EyePop.ai 第2d节点模块
EyePop.ai 节点模块为预测返回提供了方便的2d渲染功能 以 TypeScript 或 JavaScript 语言编写的应用程序推论 API.
模块要求 眼球节点 SDK
安装
节点
npm install --save @eyepop.ai/eyepop @eyepop.ai/eyepop-render-2d
浏览器
<script src="https://cdn.jsdelivr.net/npm/@eyepop.ai/eyepop/dist/eyepop.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@eyepop.ai/eyepop-render-2d/dist/eyepop.render2d.min.js"></script>
用法实例
节点
这个 EyePop 节点模块提供了 2d 渲染,用于预测 canvas.CanvasRenderingContext2D。 。 。 .
import { EyePop } from '@eyepop.ai/eyepop'
import { Render2d } from '@eyepop.ai/eyepop-render-2d'
import { createCanvas, loadImage } from 'canvas'
import { open } from 'openurl'
import { mkdtempSync, writeFileSync } from 'node:fs'
import { join } from 'node:path'
import { tmpdir } from 'node:os'
const example_image_path = 'examples/example.jpg'
;(async () => {
const image = await loadImage(example_image_path)
const canvas = createCanvas(image.width, image.height)
const context = canvas.getContext('2d')
const renderer = Render2d.renderer(context)
context.drawImage(image, 0, 0)
const endpoint = await EyePop.workerEndpoint({
pop: {
components: [
{
type: 'inference',
ability: 'eyepop.person:latest',
categoryName: 'person',
},
],
},
}).connect()
try {
let results = await endpoint.process({ source: { path: example_image_path } })
for await (let result of results) {
renderer.draw(result)
}
} finally {
await endpoint.disconnect()
}
const tmp_dir = mkdtempSync(join(tmpdir(), 'ep-demo-'))
const temp_file = join(tmp_dir, 'out.png')
console.log(`creating temp file: ${temp_file}`)
const buffer = canvas.toBuffer('image/png')
writeFileSync(temp_file, buffer)
open(`file://${temp_file}`)
})()
浏览器
<!doctype html>
<html lang="en">
<head>
<script src="https://cdn.jsdelivr.net/npm/@eyepop.ai/eyepop/dist/eyepop.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@eyepop.ai/eyepop-render-2d/dist/eyepop.render2d.min.js"></script>
</head>
<body>
<!-- ... -->
<input
type="file"
id="my-file-chooser"
/>
<!-- ... -->
<canvas id="my-canvas"></canvas>
<!-- ... -->
<script>
async uploadFile(event) {
const fileChooser = document.getElementById('my-file-chooser');
const context = document.getElementById('my-canvas').getContext("2d");
const renderer = Render2d.renderer(context);
const endpoint = await EyePop.workerEndpoint({auth: {oAuth2: true}}).connect();
await endpoint.changePop({ components: [{
type: 'inference',
model: 'eyepop.person:latest',
categoryName: 'person'
}]})
await endpoint.process({ source: {file: fileChooser.files[0]} }).then(async (results) => {
for await (let result of results) {
renderer.draw(result);
}
});
await endpoint.disconnect();
});
</script>
</body>
</html>
招标规则
默认情况下, 2d 渲染器会为预测中每个顶级对象制作框和类标签. 改变此渲染行为, 在渲染规则中通过, 例如:
// ...
Render2d.renderer(context, [Render2d.renderFace()]).draw(result)
// ...
每个规则都有 render 对象和 a target 属性。 所有预建课程都接受 JSONPath 表达式为 target 参数,用于选择哪些元素应从预测中提供.
见 JSONPath 表达式
如下表所示,大多数预建类别提供了合理的缺省.
装入相接箱和分类标签
Render2d.renderBox({
showClass: true,
showConfidence: false,
showTrackId: false,
showNestedClasses: false,
target: '$..objects.*',
})
人体病变(2d或3d)
Render2d.renderPose({
target: '$..objects[?(@.category=="person")]',
})
人手细节
Render2d.renderHand({
target: '$..objects[?(@.classLabel=="hand circumference")]',
})
人相会
Render2d.renderFace({
showLabels: false,
target: '$..objects[?(@.classLabel=="face")]',
})
渲染文本( OCR 重叠)
Render2d.renderText({
target: '$..objects[?(@.classLabel=="text")]',
})
线性分类标签
(原始内容存档于2017-10-21). Renders全相片分类标签. 显示默认 :
Render2d.renderClasses({
showConfidence: false,
labelSizePercentage: 0.05,
target: '$',
})
渲染键
给任何携带它们的物体提供关键点。 显示默认 :
Render2d.renderKeypoints({
showLabels: false,
target: '$..objects[?(@.keyPoints)]',
})
分层罩
Render2d.renderMask({
target: '$..objects[?(@.mask)]',
})
分层轮廓
Render2d.renderContour({
target: '$..objects[?(@.contours)]',
})
渲染深度地图
渲染由深度估计能力(如:. eyepop.depth.*(中文(简体) )
作为涡轮-彩色热图混合在相框上:相近=温暖(红色/黄色),相远=凉(蓝色).
天空像素( +无限) 除非 renderSky 已经设定.
Render2d.renderDepth({
target: '$',
opacity: 0.5, // overlay opacity over the underlying frame
renderSky: false, // also paint sky pixels
})
模糊对象
Render2d.renderBlur({
target: '$..objects[?(@.classLabel=="face")]',
})
注意: 这当前在对象上画上一个固态填充, 而不是应用一个真正的模糊 .
随时间推移将追踪对象拖入
Render2d.renderTrail({
target: '$..objects[?(@.traceId)]',
trailLengthSeconds: 1,
})
默认情况下,这可以追溯到对象的边框的中点. 相反,人们也可以绘制被追踪对象的子对象或关键点的痕迹. 使用可选参数 traceDetails 为此.
例如追踪每个被追踪者的鼻子:
Render2d.renderTrail({
target: '$..objects[?(@.traceId)]',
trailLengthSeconds: 1,
traceDetails: '$..keyPoints[?(@.category=="3d-body-points")].points[?(@.classLabel.includes("nose"))]',
})
自定义渲染执行
为实施自定义渲染规则,创建自定义类如下:
import { PredictedObject, StreamTime } from '@eyepop.ai/eyepop'
export interface Render {
target: string
start(context: CanvasRenderingContext2D, style: Style): void
draw(element: any, xOffset: number, yOffset: number, xScale: number, yScale: number, streamTime: StreamTime): void
}
export class RenderCustomImage implements Render {
public target: string = "$..objects[?(@.category=='person')]"
private context: CanvasRenderingContext2D | undefined
private style: any
// Optionally, add a constructor here. ie: constructor(...) { }
// The start method sets the context and style properties.
start(context: CanvasRenderingContext2D, style: any) {
this.context = context
this.style = style
}
// The draw method draws the image and line on the canvas based on the positions of the points in the PredictedObject.
public draw(
element: PredictedObject, // The object containing the keypoints
xOffset: number, // The x offset for the drawing
yOffset: number, // The y offset for the drawing
xScale: number, // The x scale factor for the drawing
yScale: number, // The y scale factor for the drawing
streamTime: StreamTime, // The timestamp for the stream of predicted objects
): void {
// Check if the context and style properties have been set
if (!this.context || !this.style) {
throw new Error('render() called before start()')
}
// Check if the keypoints are defined
if (!element.keyPoints) return
if (!element.keyPoints.length) return
// Find the points of interest in the keypoints
for (let i = 0; i < element.keyPoints.length; i++) {
const kp = element.keyPoints[i]
for (let j = 0; j < kp.points.length; j++) {
const point = kp.points[j]
if (point.classLabel === 'left eye' || point.classLabel === 'right eye') {
// Draw with the canvas context here
}
}
}
}
}