Return-Path: Delivered-To: alvarochf@traceseis.com Received: from cs510.bluehost.com by cs510.bluehost.com with LMTP id yPVYIEc0nmlDEgcAWiBQxw (envelope-from ) for ; Tue, 24 Feb 2026 16:29:11 -0700 Return-path: Envelope-to: alvarochf@traceseis.com Delivery-date: Tue, 24 Feb 2026 16:29:11 -0700 Received: from mx-out-01a.sea3.discourse.cloud ([184.105.99.121]:60125) by cs510.bluehost.com with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1vv1q3-00000001wVT-1uHV for alvarochf@traceseis.com; Tue, 24 Feb 2026 16:29:11 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=discoursemail.com; s=sea3; t=1771975740; bh=rzcrR2ud46bjnegnHY85TW3sJVARGccomv/PJxKnFs4=; h=Date:From:Reply-To:To:Subject:List-Unsubscribe:List-ID: List-Archive; b=KrkXlrI3AcBvSSpho9INgabLj4X1ghhl6oXKb8e7AN/hy3A/o0udQCo6UlNJcxbOx P6OK5ugYBsBzPqkUqRgwkONTAxlo2LHRM5SRD5aj5SLXWPrBNYb2HzEcZBSuDgZLPi mHuVqkKMeQIxylvPGPFOZyCNQ3ecW5WSkgcosD0kRTYhNSzuXB2ph/2DJoPj9VeJsl HN0rlOM+FK/dIPyQzjD991h8kNpryMvRfMKBtYKJ0iBq60DKhq4Mt8FuVBKgtcE+PK TjBAIZBnnK2I4BfhRUebSJbO77s4rKHpZQjTTIFAmT0oZvqnPGuY33NzSopa4kiL0U 9bfHDu+Xu1/Xw== Received: from localhost (unknown [IPv6:2602:fd3f:3:301:0:242:ac11:33]) by mx-out-01a.sea3.discourse.cloud (Postfix) with ESMTP id 5630713801F1 for ; Tue, 24 Feb 2026 23:29:00 +0000 (UTC) Date: Tue, 24 Feb 2026 23:29:00 +0000 From: Josafat P Reply-To: Outlier Community To: alvarochf@traceseis.com Message-ID: Subject: =?UTF-8?Q?[Outlier_Community]_[Aether_Approved_Annotators/Project?= =?UTF-8?Q?_Announcements]_=F0=9F=93=AC_Aether:_Feedback_and_Common_Errors?= =?UTF-8?Q?_Report_=28February_24=29?= MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="--==_mimepart_699e343c45823_10274d211e812578f"; charset=UTF-8 Content-Transfer-Encoding: 7bit List-Unsubscribe: List-Unsubscribe-Post: List-Unsubscribe=One-Click X-Discourse-Post-Id: 8304240 X-Discourse-Topic-Id: 2871758 X-Discourse-Tags: official urgent X-Discourse-Category: Aether Approved Annotators/Project Announcements X-Discourse-Sender: Josafat_P_QM X-Auto-Response-Suppress: All x-ms-reactions: disallow Auto-Submitted: auto-generated Precedence: list List-ID: Outlier Community | Aether Approved Annotators Project Announcements List-Archive: https://community.outlier.ai/t/aether-feedback-and-common-errors-report-february-24/2871758 Feedback-ID: outliercommunity:user_watching_first_post:discoursemail ----==_mimepart_699e343c45823_10274d211e812578f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable ## Hello, team! :smile: We would like to share some feedback, please **click on each project to s= ee the details**. [details=3D"082025-video-quality-compare"] :x: **Issues** * Annotators incorrectly chose a reconstruction with more distortion (suc= h as unclear people, trees, and signs) over a less distorted option. * Annotators incorrectly selected an inferior reconstruction when the alt= ernative maintained better quality throughout the entirety of the video a= nd was unarguably closer to the original. * Annotators incorrectly chose a distorted reconstruction that cut off pa= rt of a person's head instead of the option that was closer to the origin= al. * Annotators incorrectly marked a tie despite one reconstruction having n= oticeably worse visual quality while the alternative maintained better qu= ality throughout. :white_check_mark: **Recommendations** * A good practice is to pause the video at certain frames to directly com= pare the options against each other and against the original (A-B, A-O, B= -O). * You should look closely for any glaring differences, distortions, or mi= ssing details between the original video and the reconstructions. * The goal is to correctly identify and select the option that retains mo= re detail and is closer to the original, acknowledging that neither recon= struction may be a perfect recreation. [/details] [details=3D"260126-ud-perception-captions"] :x: **Issues** * Annotators hallucinated details for barely readable images, appearing t= o use AI generation which added unnecessary verbiage. * Annotators omitted information about the setting (like the street and s= ky), included unverifiable details (such as exact clothing colors), and m= isidentified objects (identifying a streetcar stop as a bus stop despite = visible tracks and wires). :white_check_mark: **Recommendations** * Avoid using AI tools to generate captions. * You should ensure that the details in the image are verifiable; if text= within the image is unreadable, you should clearly note it rather than g= uessing or inventing details. * A good practice is to describe the overall setting comprehensively with= out inventing details for objects that are unclear. * You should look for surrounding contextual elements, like tracks or wir= es, to accurately identify objects in the image. [/details] --- # Evals Projects (h2h) ## :warning: Main Rule for ALL Evaluations Annotators should **PRIORITIZE instruction following** and **realism/natu= ralness** above all other qualities when deciding which media is better. :memo: For example: > If Media A appears to be higher quality but does **not** follow instruc= tions as good as Media B, then **Media B should be selected**. > > > > Resolution alone should not outweigh instruction following unless it is= **explicitly requested in the prompt**. Outputs that look obviously **AI-generated** or **unnatural** should not = be preferred. Common issues to watch for include: * Overly smooth or artificial textures * Unrealistic lighting or proportions * A =E2=80=9Crendered=E2=80=9D or synthetic appearance :prohibited: An AI-generated look does **not** equal better quality. :prohibited: A visually appealing image does **not** automatically mean i= t is better quality. --- ## :bookmark: [Full document with clear examples and images (Feb 24)](htt= ps://app.outlier.ai/en/expert/guidelines/68dd6e9b668d6876a63ab1d4?compone= ntId=3D699e17097370182c64986911&type=3Dattachment) :backhand_index_pointi= ng_left: --- > You should focus on technical precision and **creative** execution acro= ss several key project areas. > :light_bulb: We encourage you to implement the creative tips posted pre= viously [here](https://community.outlier.ai/t/creativity-tips/2858014). --- As a reminder of the grading scale: * :-1: =E2=80=9C**Major Issues**=E2=80=9D feedback means the task contain= s major errors or misalignment. * :information_source: =E2=80=9C**Some Issues**=E2=80=9D feedback means t= he task contains minor omissions/errors. * :+1: =E2=80=9C**Meeting Expectations**=E2=80=9D feedback means the task= meets all requirements (common and expected). * :star: "**Exceptional Work**=E2=80=9D feedback means you are doing grea= t! Your task can be used as a stellar example and exceeds expectations (t= his is rare). --- [Visit Topic](https://community.outlier.ai/t/aether-feedback-and-common-e= rrors-report-february-24/2871758/1) or reply to this email to respond. To unsubscribe from these emails, [click here](https://community.outlier.= ai/email/unsubscribe/a42bf3d113936b6c1dd9641d3e5811f92a17d0de94f215da8225= c7c3fcdd62f5). ----==_mimepart_699e343c45823_10274d211e812578f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable = =
= =
Josafat_P_QM
February 24

Hello, tea= m! 3D":smile:"

We would like to share some feedback, = please click on each project to see the details.

082025-video-quality-compare (click for more detail= s) 260126-ud-perception-captions (click for more detai= ls)

Evals Projects (h2h)

3D":warning:" Mai= n Rule for ALL Evaluations

Annotators should PRIORITIZE i= nstruction following and realism/naturalness ab= ove all other qualities when deciding which media is better.

3D":m= For example:

If Media A appears to be hig= her quality but does not follow instructions as good as = Media B, then Media B should be selected.

Resolution a= lone should not outweigh instruction following unless it is expli= citly requested in the prompt.

Outputs that look obviously AI= -generated or unnatural should not be preferred= . Common issues to watch for include:

  • Overly smooth or artificial textures
  • Unrealistic lighting or proportions
  • A =E2=80=9Crendered=E2=80=9D or synthet= ic appearance

3D":prohibited:" An A= I-generated look does not equal better quality.
3D":prohibited:" A visually appealing image does not automatically mean it is better quality.

3D":bookmark:" Full document with clear examples and images (Feb 24) <= img src=3D"https://emoji.discourse-cdn.com/twitter/backhand_index_pointin= g_left.png?v=3D15" title=3D":backhand_index_pointing_left:" class=3D"emoj= i" alt=3D":backhand_index_pointing_left:" loading=3D"lazy" width=3D"20" h= eight=3D"20">

You should f= ocus on technical precision and creative execution acros= s several key project areas.
3D":light_bulb:" We encourage you to implement the creat= ive tips posted previously here.

As a reminder of the grading scale:

  • 3D":-1:" =E2=80=9CMajor Issues= =E2=80=9D feedback means the task contains major errors or misal= ignment.
  • 3D":information_source:" =E2=80=9CSome Issues=E2=80=9D feedbac= k means the task contains minor omissions/errors.
  • 3D":+1:" =E2=80=9CMeeting Expe= ctations=E2=80=9D feedback means the task meets all requirements= (common and expected).
  • 3D":st= "Exceptional Wo= rk=E2=80=9D feedback means you are doing great! Your task can be= used as a stellar example and exceeds expectations (this is rare).

Visit Topic or reply t= o this email to respond.

To unsubscribe from these= emails, click here.

                  &nbs= p;                   &nbs= p;                   &nbs= p;
= = = = ----==_mimepart_699e343c45823_10274d211e812578f--