Determine corner points of marker in camera frame without opengl

Community support forum for the ARToolKit v5.x for Android SDK.
ekkelenkamp
Posts: 10
Joined: Mon Aug 15, 2016 12:44 pm

Determine corner points of marker in camera frame without opengl

Postby ekkelenkamp » Mon Aug 15, 2016 12:54 pm

Hi,

I'm building an Android app without using opengl.
I would like to determine the corner points of the detected markers in the camera frame that I pass to arToolkit.

I'm able to detect the markers and using the queryMarkerTransformation method I can get a projectionMatrix:

Code: Select all

float[] projectionMatrix = arToolKit.queryMarkerTransformation(markerId);


It's unclear to me how I can use the projectionMatrix to determine the marker position on the camera frame. The sample code only shows how to render 3d objects with opengl using this matrix.

I only want to use ARToolkit for detecting the markers, no 3d rendering is required.

Any help would be appreciated.

Regards,

Rudie.

User avatar
philip_lamb
Site Admin
Posts: 664
Joined: Thu Mar 09, 2006 8:35 pm

Re: Determine corner points of marker in camera frame without opengl

Postby philip_lamb » Mon Aug 15, 2016 1:47 pm

Hi Rudie,

Actually, the corner point locations are available at an earlier stage in the processing, however this is not currently exposed in the Android Java API. From the C header <AR/ar.h>:

Code: Select all

/*!
    @typedef ARMarkerInfo
    @abstract   Describes a detected trapezoidal area (a candidate for a marker match).
    @discussion
        After marker detection, a number of trapezoidal areas in the camera image will have been identified. An
        ARMarkerInfo struct is returned for each area so matched. Trapezoidal areas which have been matched
        with marker images (in pattern mode) or barcodes (in matrix mode) will have valid values assigned to the
        appropriate id field.
    @field      area Area in pixels of the largest connected region, comprising the marker border and regions connected to it. Note that this is
        not the same as the actual onscreen area inside the marker border.
   @field      id If pattern detection mode is either pattern mode OR matrix but not both, will be marker ID (>= 0) if marker is valid, or -1 if invalid.
   @field      idPatt If pattern detection mode includes a pattern mode, will be marker ID (>= 0) if marker is valid, or -1 if invalid.
    @field      idMatrix If pattern detection mode includes a matrix mode, will be marker ID (>= 0) if marker is valid, or -1 if invalid.
   @field      dir If pattern detection mode is either pattern mode OR matrix but not both, and id != -1, will be marker direction (range 0 to 3, inclusive).
   @field      dirPatt If pattern detection mode includes a pattern mode, and id != -1, will be marker direction (range 0 to 3, inclusive).
   @field      dirMatrix If pattern detection mode includes a matrix mode, and id != -1, will be marker direction (range 0 to 3, inclusive).
   @field      cf If pattern detection mode is either pattern mode OR matrix but not both, will be marker matching confidence (range 0.0 to 1.0 inclusive) if marker is valid, or -1.0 if marker is invalid.
   @field      cfPatt If pattern detection mode includes a pattern mode, will be marker matching confidence (range 0.0 to 1.0 inclusive) if marker is valid, or -1.0 if marker is invalid.
   @field      cfMatrix If pattern detection mode includes a matrix mode, will be marker matching confidence (range 0.0 to 1.0 inclusive) if marker is valid, or -1.0 if marker is invalid.
   @field      pos 2D position (in camera image coordinates, origin at top-left) of the centre of the marker.
   @field      line Line equations for the 4 sides of the marker.
   @field      vertex 2D positions (in camera image coordinates, origin at top-left) of the corners of the marker. vertex[(4 - dir)%4][] is the top-left corner of the marker. Other vertices proceed clockwise from this. These are idealised coordinates (i.e. the onscreen position aligns correctly with the undistorted camera image.)
   @field      markerInfo2Ptr (description)
    @field      cutoffPhase If a trapezoidal region is detected, but is eliminated from the candidates for tracking,
        this field is filled out with the tracking phase at which the marker was cut off. An English-language
        description of the phase can be obtained by indexing into the C-string array arMarkerInfoCutoffPhaseDescriptions[].
    @field      globalID If arPattDetectionMode is a matrix mode, matrixCodeType is AR_MATRIX_CODE_GLOBAL_ID, and idMatrix >= 0, will contain the globalID.
*/
typedef struct {
    int             area;
    int             id;
    int             idPatt;
    int             idMatrix;
    int             dir;
    int             dirPatt;
    int             dirMatrix;
    ARdouble        cf;
    ARdouble        cfPatt;
    ARdouble        cfMatrix;
    ARdouble        pos[2];
    ARdouble        line[4][3];
    ARdouble        vertex[4][2];
    ARMarkerInfo2  *markerInfo2Ptr;
    AR_MARKER_INFO_CUTOFF_PHASE cutoffPhase;
    int             errorCorrected;
    uint64_t        globalID;
} ARMarkerInfo;

During marker detection, ARMarkerInfo is retrieved by the call arGetMarker. Do a search in the C source for ARWrapper and you can see this.

It should be possible to expose the values via JNI with a little work.

ekkelenkamp
Posts: 10
Joined: Mon Aug 15, 2016 12:44 pm

Re: Determine corner points of marker in camera frame without opengl

Postby ekkelenkamp » Mon Aug 15, 2016 10:37 pm

Hi Philip,

Thanks for your response and the good suggestion. I'll try to get the information with JNI. Is there any chance this information will be exposed in the official java api of artoolkit? This would really help integrating with other toolkits. Currently I'm using opencv that lack support for marker detection.

Regards,

Rudie.

ekkelenkamp
Posts: 10
Joined: Mon Aug 15, 2016 12:44 pm

Re: Determine corner points of marker in camera frame without opengl

Postby ekkelenkamp » Wed Aug 17, 2016 12:03 pm

Hi,

I succeeded in getting the corner point by adjusting some of the wrapper code. It works great!
Would it be useful to create a pull request for this change?

What I did is the following:

Add an entry to ARMarker.h

Code: Select all

    float cornerPoints[8];


Extended ARMarkerSquare.cpp and in the updateWithDetectedMarkers method just after the code where a marker has been determined visible, update the cornerPoints:

Code: Select all

   // Consider marker visible if a match was found.
        if (k != -1) {
            visible = true;
            m_cf = markerInfo[k].cf;
            for (int c = 0; c < 4; c++) {
                cornerPoints[c*2] = markerInfo[k].vertex[c][0];
                cornerPoints[c*2 + 1] = markerInfo[k].vertex[c][1];
            }



Added a new method ARToolKitWrapperExportedAPI.cpp to retrieve the corner points:

Code: Select all

EXPORT_API bool arwQueryMarkerCornerPoints(int markerUID, float points[8])
{
    ARMarker *marker;

    if (!gARTK) return false;
    if (!(marker = gARTK->findMarker(markerUID))) {
        gARTK->logv(AR_LOG_LEVEL_ERROR, "arwQueryMarkerCornerPoints(): Couldn't locate marker with UID %d.", markerUID);
        return false;
    }
    for (int i = 0; i < 8; i++) points[i] = (float)marker->cornerPoints[i];
    return marker->visible;
}


And added the JNI definition for that:

Code: Select all



JNIEXPORT jfloatArray JNICALL JNIFUNCTION(arwQueryMarkerCornerPoints(JNIEnv *env, jobject obj, jint markerUID))
{
    float trans[8];

    if (arwQueryMarkerCornerPoints(markerUID, trans)) return glArrayToJava(env, trans, 8);
    return NULL;
}



After all that I recompiled the ARWrapper shared objects in the android directory and used these new methods.

In the he NativeInterface.java I added the following method:

Code: Select all

    /**
     * Retrieves the corner points for the specified marker
     *
     * @param markerUID The unique identifier (UID) of the marker to check
     * @return A float array of size 8 containing the corner points starting at top left (x,y) top right, bottom right, bottom left.
     * So
     */
    public static native float[] arwQueryMarkerCornerPoints(int markerUID);



And finally add the method to ARToolKit.java as well:

Code: Select all


    /**
     * Retrieves the corner points for the specified marker
     *
     * @param markerUID The unique identifier (UID) of the marker to check
     * @return A float array of size 8 containing the corner points starting at top left (x,y) top right, bottom right, bottom left.
     *
     */
    public float[] arwQueryMarkerCornerPoints(int markerUID) {
        if (!initedNative) return null;
        return NativeInterface.arwQueryMarkerCornerPoints(markerUID);
    }
   

MrDaniel
Posts: 381
Joined: Wed Nov 18, 2015 9:07 pm

Re: Determine corner points of marker in camera frame without opengl

Postby MrDaniel » Sun May 21, 2017 1:55 pm

Hello.

Please create a PR and i will fgegt it reviewed by the dev team.

Regards,

Daniel


Return to “ARToolKit for Android”

Who is online

Users browsing this forum: No registered users and 2 guests