Full text for verification
AI Learning - Decentralized Governance to Optimize Human Output Datasets for AI Learning
Canonical record: https://ssrn.com/abstract=4755632
42 protected claims are extracted from this work.
Source extraction SHA-256: 9abd393a95cb5c1adf20197f800ba77ae6397765d3ca480fede9a5fe88930773
# **AI Learning**
### **Decentralized Governance to Optimize Human Output Datasets for AI Learning**
```
WulfKaal,Ph.D.1
```
## **`Abstract`**
```
TheevolutionofAIdependsonupgradablequalitydatasets.Data
isthefoundationonwhichAIalgorithmslearnandmake
predictions.High-quality,diverse,andlabeleddatasetsare
crucialfortrainingAImodelseffectively.Theavailabilityof
qualitydataplaysasignificantroleindeterminingthesuccess
andimpactofAIindisruptedindustries.
```
```
TheAILearningEcosystem(ALE)facilitatesamicrotask
ecosystemforAIlearning.ALEusesitsprovenandtested
decentralizedgovernanceecosystemtoprovidehigh-quality
diversedatasetsforAIlearningviagamifiedmicro-taskwork.
Throughitstestingenvironmentintheindustry-leadingCode
ReviewDAO(CRDAO),ALEdistinguishesitselffromcompetitors
throughunparalleleddecentralizedgovernanceoptimizationthat
minimizesmicro-taskworkduplicationincentralizedsystemsand
allowsgamifiedmicro-taskworktoscalehigh-qualitydiverse
datasetsforAIlearning.
```
```
KeyWords:ArtificialIntelligence,LargeLanguageModels,
Dataset,MicroTaskWork,Gamification,QualityControls,
DecentralizedAutonomousOrganization,TokenModels,
Cryptocurrencies,FeedbackEffects,EmergingTechnology,Tokens,
Blockchain,DistributedLedgerTechnology,CodeAssurances
```
```
JELCategories:K20,K23,K32,L43,L5,O31,O32
```
> 1 `The Author is grateful to many members of ALE and the open source community who provided valuable feedback.`
# **`Table of Contents`**
|**Background..................................................................................................................................3**|
|---|
|AI for Human Evolution............................................................................................................3|
|AI Business Model Disruption and Scaling..............................................................................4|
|Pain Point Data Quality............................................................................................................6|
|Total Addressable Market........................................................................................................ 8|
|Disrupted Industries...........................................................................................................9|
|Data Providers................................................................................................................. 14|
|**Gamification to enhance AI Data Quality................................................................................ 15**|
|Freelance Micro Task Market Growth....................................................................................16|
|Micro Task Work for AI Learning............................................................................................18|
|Shortcomings in Legacy Micro Task Market.......................................................................... 20|
|Lack of Scaling Solutions in Centralized Micro Task Market............................................21|
|Overpricing.................................................................................................................21|
|Duplication of Work....................................................................................................22|
|No Access for Workers.............................................................................................. 23|
|Gamification to Enhance Micro Task Work............................................................................ 23|
|**Case Study - Code Review Platform........................................................................................26**|
|The Code Review Platform is operated by an affiliated entity and served as a testing<br>environment in the past. It is accessible under the following URL: https://crdao.ossa.dev/.. 26|
|Community Audit for Code Review........................................................................................26|
|**AI Learning Ecosystem Platform............................................................................................. 46**|
|Scaling the Code Review Platform with Micro Task Work..................................................... 46|
|Removing Cost of Micro Task Work Duplication....................................................................46|
|Transaction Cost Minimization of Micro Task Work............................................................... 49|
|Gamification of Micro Task Work........................................................................................... 49|
# **`Background`**
# **AI for Human Evolution**
```
ArtificialIntelligence(AI)epitomizeshumanevolution.Based
onfindingsassociatedwithhumanbrainfunctionality,deep
learninginAIsystemsisamethodofproducingAIcomponents
thatcanprocessdatasimilartohowthehumanbrainwould
processdata.ThissupportstheAI’srecognitionofcomplex
patternsindatasets,includinginpictures,textelements,and
sounds.Theanalysisofsuchdatasetsindeeplearningallows
theAItogenerateinsights,analysis,andpredictionsfora
varietyofhumangeneratedoutcomes.ThroughdeeplearningAI
canautomatesequencesoftasksthatotherwisewouldrequire
humanintelligence.Examplesmayincludetheanalysisandcausal
recognitionofimagesortranscribingasoundfileintotext,
amongothers.
```
```
In2024,AIdeeplearningispossiblethroughsocalledLarge
languagemodels(LLM).LLMsaretrainedonlargedatasets.For
thispurpose,neuralnetworks,throughtheirencodersand
decodersextracthumanlevelmeaningsfromasequenceofdata
includingtextandcreateassociatedrelationshipsbetweenwords
andphrasesinthetextthatiscontainedinthelargedatasets.
```
```
So-calledtransformerneuralnetworkarchitectureenablesthe
utilizationoflargedatasetswithuptohundredsofbillionsof
parameters.Whiletransformerneuralnetworkarchitecturecan
utilizehugedatasetsthatareoftengeneratedfrominternet
data,withoftenover50billionwebpages,andWikipedia,with
itsevergrowingamountofpages,dataqualitycontinuestobea
hugeissueforLLMs.
```
# **AI Business Model Disruption and Scaling**
```
DeeplearningAIhasunparalleledusecasesthatallowan
industry-widescalingofnewbusinessmodelsbuiltonAI,
includinginelectricvehicles,legal,administrative,
manufacturing,aerospace,electronics,medicalresearch,and
```
```
manyotherindustriesandfields.
```
```
AIhasthepotentialtodisruptexistingbusinessmodelsand
enablethecreationofnewandinnovativebusinessesinseveral
ways:
```
```
1.Automationandefficiency:AIcanautomaterepetitiveand
mundanetasks,leadingtoincreasedefficiencyandcostsavings.
Thisdisruptioncaneliminatetheneedforcertainjobrolesor
transformthem,allowingbusinessestoallocateresourcesmore
effectivelyandfocusonhigher-valueactivities.
```
```
2.Enhanceddecision-making:AIalgorithmscananalyzevast
amountsofdataquicklyandaccurately,providingvaluable
insightsfordecision-making.Thisdisruptionempowers
businessestomakedata-drivendecisions,optimizeprocesses,
andimproveoutcomes.
```
```
3.Personalizationandcustomerexperience:AIenables
businessestopersonalizetheirproducts,services,andcustomer
experiencesbasedonindividualpreferencesandbehavior.By
leveragingAIalgorithms,businessescancreatetailored
recommendations,targetedmarketingcampaigns,andpersonalized
interactions,enhancingcustomersatisfactionandloyalty.
```
```
4.Predictiveanalytics:AIalgorithmscanpredictfuture
trends,customerbehavior,andmarketdynamicsbyanalyzing
historicaldata.Thisdisruptionallowsbusinessestoanticipate
demand,optimizeinventory,mitigaterisks,andmakeproactive
decisions,givingthemacompetitiveadvantage.
```
```
5.Newbusinessmodelsandservices:AIopensdoorstoentirely
```
```
newbusinessmodelsandservices.Forexample,AI-powered
platformscanconnectconsumersandprovidersdirectly,
disruptingtraditionalintermediaries.Additionally,AIcan
enablethecreationofinnovativeproductsandservicesthat
werenotpossiblebefore,suchasautonomousvehicles,virtual
assistants,orpersonalizedhealthcaresolutions.
```
```
6.Scalabilityandscalability:AIsystemscanscaleefficiently
andhandlelargevolumesofdataandinteractions,makingit
easierforbusinessestoexpandtheiroperationsrapidly.This
disruptionallowsbusinessestoreachbroadermarketsandserve
alargercustomerbasewithoutsignificantinfrastructure
investments.
```
# **`Pain Point Data Quality`**
```
ThegrowthandsuccessofAIarehighlydependentonquality
datasets.DataisthefuelthatpowersAIalgorithms,and
withouthigh-quality,relevant,anddiversedata,the
performanceandaccuracyofAIsystemscanbecompromised.
```
```
Whileitisimportanttonotethatdataqualityaloneisnot
sufficientforAIgrowthanditsdisruptionandreinventionof
businessmodels,thealgorithms,computingpower,andexpertise
ofdevelopersalsoplaysignificantroles.Nonetheless,access
toqualitydatasetsisafundamentalrequirementforthe
developmentandadvancementofAItechnologies.
```
```
Thekeyreasonswhyhighqualitydatasetsarecrucialforthe
AIevolutionincludebutarenotlimitedtothefollowing:
```
**`1. Training AI models:`** `AI systems learn from data through a`
```
processcalledtraining.Duringtraining,AIalgorithmsanalyze
andextractpatternsfromlargedatasetstomakepredictionsor
performspecifictasks.Thequalityofthetrainingdata
directlyimpactstheAImodel'sabilitytolearnandmake
accuratepredictions.
```
```
2.Biasreduction:High-qualitydatasetshelpinreducingbias
```
```
withinAIsystems.Biaseddatacanleadtobiasedoutcomes,
perpetuatingsocialorculturalprejudices.Byensuringdiverse
andrepresentativedatasets,AIdeveloperscanworktowards
minimizingbiasandcreatingfairerAIsystems.
```
```
3.Generalizationandadaptability:QualitydatasetshelpAI
modelsgeneralizetheirlearningtonewsituationsandadaptto
changingenvironments.Adiverseandcomprehensivedataset
allowsAIsystemstoencounterawiderangeofscenarios,
leadingtobetterperformanceinreal-worldapplications.
```
```
4.Robustnessandreliability:AImodelstrainedonhigh-quality
datasetstendtobemorerobustandreliable.Theycanhandle
edgecases,outliers,andunexpectedinputswithgreater
accuracy,improvingtheoverallperformanceanduserexperience.
```
```
In2024,AIdevelopmentcompaniesarebeginningtousesmaller
datasets2forthedevelopmentofLLMs.Thisisdoneinaneffort
toincreasemodelaccuracyandpreventoverfittinginmachine
learningmodels,which,inturn,allowsthenewmachinelearning
modelstoproducebetterresultsonunseendata.However,with
smalldatasetsinLLMs,theriskofoverfittingalsorises,
especiallywithcomplexmodels.Therefore,LLMdevelopershave
toturntoregularizationinanefforttoaddressoverfittingof
themodelwiththetrainingdata.
```
```
BecauseofthetrendtowardssmallerdatasetsinLLMs,data
qualityisacrucialpainpointinsmallerdatasets.Inaddition
tobalancingandnormalizingthedata,LLMdevelopersfocuson
adequatemodelvalidationthroughtechniquesincludingdropouts
inneuralnetworks,pruningdecisiontrees,and
cross-validation.Alloftheseeffortsaredevotedtogenerating
bettertrainingdatasetsasthequalityofLLMtrainingdatacan
breakLLMresultsorimprovethemsignificantly.
```
# **Total Addressable Market**
```
Industriesacrosstheboardcanbesignificantlyaffectedbythe
availabilityofhigh-qualityAItrainingdatasets.Some
industriesmayexperienceamorepronouncedimpactduetothe
```
2 https://towardsdatascience.com/is-small-data-the-next-big-thing-in-data-science-9acc7f24907f
```
natureoftheiroperationsandthepotentialforAI-driven
```
```
transformations.
```
# Disrupted Industries
```
Estimatingthetotaladdressablemarket(TAM)forindustries
impactedbytechnologicaladvancements,particularlyartificial
intelligence(AI),isamultifacetedtask.Itrequiresanalyzing
varioussectors,eachwithitsownmarketsizeandgrowth
trajectory.ThedynamicnatureofAI'sinfluencecomplicatesthe
efforttopinpointanexactmarketvalue.Nonetheless,the
aggregatepotentialmarketsizeisundeniablyvast.
```
```
TocontextualizetheTAMforindustriesundergoingdisruption,
```
```
considerthefollowingsectors:
```
## **`1. Healthcare:`**
```
AIapplicationsinhealthcare,suchasdiseasediagnosis,drug
discovery,personalizedmedicine,andpatientmonitoring,depend
onhigh-qualitydatasetsforaccuracy.Theseapplicationsare
poisedtoenhancemedicaldecision-makingandpatientcare
significantly.AccordingtotheWorldHealthOrganization,the
globalhealthcaremarketwasestimatedat$8.45trillionin2020
```
```
andisprojectedtogrowto$11.91trillionby2025,
```
```
highlightingAI'scriticalroleinthissector'sevolution.3
```
# **`2. Finance:`**
```
Thefinancesectorreliesondataforfunctionsincludingrisk
assessment,frauddetection,andalgorithmictrading.AImodels
thatprocessthisdatacanuncoverpatternsandpredictmarket
movements,therebyfacilitatingsmarterinvestmentstrategies
andcustomerserviceimprovements.Theglobalfinancialservices
market'svaluestoodatabout$19.7trillionin2020,withAI
expectedtotransformbanking,insurance,andassetmanagement.4
```
# **`3. Retail and E-commerce:`**
```
Inretailande-commerce,AI-driventoolsforpersonalization,
demandforecasting,andinventorymanagementrelyondetailed
dataanalysis.Thesetoolsenablebusinessestotailortheir
offeringsandoptimizeoperations.Theglobalretailmarket's
valuationexceeded$22trillionin2020,andAI'scontributions
aresettoredefinetheindustrybyboostinggrowthand
```
```
enhancingtheconsumerexperience.5
```
> 3 `"Global Spending on Health: A World in Transition," World Health Organization, 2021.`
4 `"Global Financial Markets: An Overview," International Monetary Fund, 2020.`
> 5 `"The State of the Global Retail Market," World Retail Congress, 2020.`
# **`4. Manufacturing and Logistics:`**
```
AIenhancesmanufacturingandlogisticsthroughimproved
efficiency,qualitycontrol,andsupplychainmanagement.The
globalmanufacturingmarketwasvaluedataround$39.6trillion
in2020.AIapplicationsinthissectorpromisesignificantcost
reductionsandperformanceenhancements.6
```
# **`5. Transportation:`**
```
AI'spotentialintransportation,includingautonomousvehicles
andtrafficmanagement,isconsiderable.Theglobal
transportationservicesmarketwasvaluedatapproximately$4.3
trillionin2020.AsAItechnologiessuchasautonomousdriving
androuteoptimizationadvance,theTAMforAIintransportation
isexpectedtowitnesssubstantialgrowth.7
```
# **`6. Customer Service:`**
```
ThetransformationofcustomerservicebyAI,throughchatbots
andvirtualassistants,underscoresthetechnology'svaluein
enhancingcustomerinteractions.Thecustomerservicesector,
valuedatabout$55billionin2020,willseeitsTAMforAI
```
> 6 `"Global Manufacturing Outlook," United Nations Industrial Development Organization, 2021.`
> 7 `"Global Transportation Market Analysis," International Transport Forum, 2020.`
```
expandasbusinessesincreasinglyadoptthesesolutionsto
```
```
improveefficiencyandcustomersatisfaction.8
```
```
Thesesectorsexemplifythesignificant,albeitchallengingto
quantify,TAMforAIacrossindustries.Theongoingintegration
ofAIintotheseareaspromisesnotonlytodrivemarketgrowth
butalsotocatalyzeprofoundchangesinhowservicesare
deliveredandconsumedworldwide.
```
```
Theimpactofhigh-qualityAItrainingdatasetsisnotlimited
totheseabovementionedindustries.AIhasthepotentialto
disruptandtransformvarioussectors,includingeducation,
energy,agriculture,entertainment,andmore.Theavailability
ofqualitydatasetsisacatalystforAI-drivenadvancements
andinnovationacrossindustries.
```
```
PleasenotethattheseTAMfiguresareapproximateandsubject
tovariousfactorssuchasmarketdynamics,technological
advancements,andglobaleconomicconditions.Thegrowth
potentialofAIintheseindustriesissignificantandwill
continuetoevolveasthetechnologymaturesandadoption
increases.
```
> 8 `"Global Customer Service Market Report," Customer Service Institute, 2020.`
```
However,acquiringandcuratinghigh-quality,diverse,and
labeleddatasetscanbeachallenge.Industriesthatcanaccess
largevolumesofstructuredandlabeleddata,suchasfinance
andhealthcare,mayhaveaheadstartinleveragingAI.Onthe
otherhand,industriesliketransportationandcustomerservice
mayfacehurdlesinobtainingqualitydataduetoprivacy
concerns,datafragmentation,andtheneedforextensive
labelingefforts.Overcomingthesechallengesandensuring
accesstoqualitydataisessentialformaximizingthepotential
TAMofAIinthesesectors.
```
```
Despitethesechallenges,theTAMforAIintransportationand
customerserviceissubstantial.AsAItechnologiesmature,data
availabilityimproves,andcompaniesinvestinAI-driven
solutions,theTAMfortheseindustriesisexpectedtoincrease
significantly.ThecombinedTAMofalldisruptedindustries,
includinghealthcare,finance,retail,manufacturing,logistics,
transportation,andcustomerservice,islikelytobeinthe
hundredsoftrillionsofdollars,makingitahighlylucrative
marketforAI-driveninnovation.AddingtheTAMofother
affectedsectorslikeeducation,energy,agriculture,and
entertainment,thetotaladdressablemarketforalldisrupted
```
```
industriescombinedisinthehigherhundredsoftrillionsof
```
```
dollars.
```
# Data Providers
```
DeterminingtheexactTAMfordataproviderswhoproduce
high-qualitydiversedatasetsforAIlearningischallengingas
itdependsonvariousfactorssuchasmarketdemand,pricing,
andcompetition.However,itissafetosaythattheTAMfor
suchdataprovidersissignificantandgrowingrapidly.
```
```
AsAIcontinuestoadvanceandbecomemoreprevalentacross
industries,theneedforqualitytrainingdatabecomes
paramount.Dataprovidersthatcandeliverlabeled,diverse,and
reliabledatasetstotrainAImodelseffectivelywillhavea
```
```
significantmarketopportunity.
```
```
TheTAMfordataprovidersinAIlearningextendsacross
multiplesectors,includinghealthcare,finance,retail,
manufacturing,logistics,transportation,customerservice,
education,energy,agriculture,andentertainment,amongothers.
Eachoftheseindustriesrequiresspecificdatasetstailoredto
```
```
theiruniqueneedsandusecases.
```
```
Furthermore,withtheincreasingadoptionofAIbybothlarge
enterprisesandsmallerbusinesses,thedemandforhigh-quality
dataisexpectedtosurge.Startups,establishedcompanies,and
evenAIserviceproviderswillrelyondataproviderstoaccess
thenecessarydatasetsfortheirAIinitiatives.
```
```
ConsideringthepotentialTAMofthedisruptedindustries
mentionedearlier,whichisestimatedtobeinthehundredsof
trillionsofdollars,theTAMfordataproviderswhoproduce
high-qualitydiversedatasetsforAIlearningislikelytobe
substantial.AsAIbecomesmoreintegraltobusinessoperations
anddecision-makingprocesses,theimportanceofquality
trainingdatawillonlygrow,furtherexpandingtheTAMfordata
providersinthisspace.
```
```
GamificationtoenhanceAIDataQuality
```
```
TheAILearningEcosystem(ALE)usesitsprovenandtested
decentralizedgovernanceecosystemtoprovidehigh-quality
```
```
diversedatasetsforAIlearningviamicro-taskwork.
```
```
ThroughitstestingenvironmentintheexistingCodeReviewDAO,
```
```
ALEdistinguishesitselffromcompetitorsthrough
```
```
industry-leadingdecentralizedgovernanceoptimizationthat
```
```
minimizesmicro-taskworkduplicationincentralizedsystemsand
allowsmicrotaskworktoscaleforAIdatasetgeneration.
```
# **`Freelance Micro Task Market Growth`**
```
Inthecurrentfreelancermarket,leadingcentralizedplatforms
Fiverr,Taskrabbit,Upwork,CrowdflowerandMechanicalTurkhave
theirworkforcesdistributedthroughouttheworld.MicroTasks
freelanceworkisbecomingincreasinglyimportantinthe
freelancemarket.
```
```
Microtasksaredefinedassmalltasksthatrequirehuman
judgment,canbecompletedbyhumansindependentlyoverthe
internet,andarepartofalargerunifiedproject.Becauseof
thenecessityofhumanjudgmentthatcannotcurrentlybe
replacedbymachines/computers,microtasksenableorganizations
tobuildproductsorcreateoutcomesthatcannotbesynthesized
bymachines/computersalone.Forexample,theChinesegovernment
uses2millionmicrotaskworkerstoaidincensoringthe
internet.Internetcompanies,suchasGoogle,Facebook,Twitter,
Ebay,andLinkedinoptimizetheirproductionreadysolutionsand
enhancetheirAImodeltrainingwithmicrotaskworkers.Large
scaledistributionsofmachinelearningresearchers,among
```
```
others,gatherstructuredlabeleddataforartificial
intelligence(AI)trainingpurposesthroughtheuseofmicro
```
```
taskworkers.
```
```
Billionsofmicrotasksarecompletedeachyearandthedemand
formicrotaskworkersisincreasingconsistently.Inanattempt
tocapitalizeonthegrowingdemandformicrotaskwork,Amazon
createdAmazonMechanicalTurk(MTurk)in2005.MTurkisan
onlinemarketplacethatallowsrequesterstopayworkersfor
performingmicrotasksonline,thuscrowdsourcingdata
collection.TheWorldBankconcludedina2015report,thatthe
largestcrowdsourceddatacollectionplatformsAmazonMechanical
TurkandCrowdFlower,aventurebackedcompanythatraisedover
$58millionandisfocusedonenrichingdatausedforAI,would
quadrupletheirrevenuefrom2013to2016.
```
```
Theincreasingdemandformicrotaskworkersthatcameinitially
fromdatascientistsandotheracademicsisfurtherincreasedby
thegrowthofartificialintelligence(AI)andtheincreasing
scopeandscaleofAIapplicationsinabroadrangeof
industries.Fortune500techcompaniesincludingAmazon,Apple,
GoogleAlphabet,Twitter,andFacebookuseAIandmachine
learningtoimprovetheirservicesandcutcosts,thus
increasingprofitabilityexponentially.Accordingtosome
```
```
estimates,Fortune500techcompaniesspentbetween$20and$30
```
```
billiononthedevelopmentandenhancementoftheirAIsystems
in2016.Morerecentdatafor2017suggeststhatthistrendis
continuingandincreasing.AIisalreadyplayingasignificant
roleinconsumerexpectationsandtechcompanieshavethebest
possiblesetofincentivestokeepinvestinginAItofulfill
```
```
suchexpectationsanddevelopnewproductsatmarginalcost.
```
# **Micro Task Work for AI Learning**
```
Microtaskworkplaysacrucialroleinproducinghigh-quality
anddiversedatasetsforAIlearning.Itinvolvesbreakingdown
complextasksintosmaller,moremanageabletasksthatcanbe
completedbyalargenumberofhumanworkers.Theseworkers,
oftenreferredtoascrowdworkersormicrotaskers,perform
thesetasksinexchangeformonetarycompensation.
```
```
Thefollowingexamplesshowhowmicrotaskworkcontributesto
thecreationofhigh-qualitydatasetsforAIlearning:
```
```
1.DataAnnotation:Microtaskworkiscommonlyusedfordata
annotationtasks,wherecrowdworkerslabelandannotatedata
accordingtospecificcriteria.Forexample,inimage
```
```
recognition,crowdworkersmayannotateobjectsordrawbounding
boxesaroundthemtotrainAImodels.Thisannotationprocess
helpscreatelabeleddatasetsthatserveasgroundtruthfor
trainingAIalgorithms.
```
```
2.DataValidation:Crowdworkersalsoplayacriticalrolein
validatingthequalityandaccuracyoflabeleddata.Theycan
reviewandverifyannotationsmadebyotherworkers,ensuring
consistencyandreducingerrors.Thisvalidationstephelps
```
```
maintaintheintegrityandreliabilityofthedataset.
```
**`3. Data Augmentation:`** `Micro task work can be utilized to`
```
generatediversedatabycreatingvariationsoraugmentationsof
existingdatasets.Crowdworkerscanperformtaskslikeimage
manipulation,textparaphrasing,oraudiosynthesistoincrease
```
```
thediversityofthetrainingdata.ThishelpsAImodels
generalizebetterandperformwellonawiderrangeof
```
```
real-worldscenarios.
```
```
4.DataCleaning:Crowdworkerscanassistincleaningand
refiningdatasetsbyidentifyingandrectifyingerrors,removing
duplicates,orstandardizingdataformats.Thisensuresthe
datasetisofhighquality,reducingnoiseandimprovingthe
performanceofAImodelsduringtrainingandinference.
```
```
Microtaskworkplatforms,suchasAmazonMechanicalTurk,
FigureEight(nowAppen),andScaleAI,providethe
infrastructuretodistributethesetaskstoalargepoolof
workers,managetheircontributions,andensurequalitycontrol.
```
```
Overall,microtaskworkenablestheefficientandscalable
productionofhigh-quality,diversedatasetsforAIlearning.It
leverageshumanintelligencetohandletasksthatare
challengingforcurrentAIsystemsandcontributestothe
continuousimprovementandadvancementofAItechnologies.
```
# **`Shortcomings in Legacy Micro Task Market`**
```
Theevolution,improvements,andgrowthofAIiscorrelatedwith
theevolution,improvements,andgrowthinmicrotaskwork.AI
usessupervisedandunsupervisedaswellasreinforcement
machinelearning.Becauseunsupervisedandreinforcement
learningaremuchmorecomplexthansupervisedlearning,
supervisedlearningistodatemorecommonandmorereliedupon
forAIdevelopment.Whilethismaychangeasunsupervisedand
reinforcementlearningevolve,currently,supervisedlearning
```
```
dependsonlabeleddatathatisproducedviamicrotaskwork.
ThemappingfunctionofthesupervisedAIlearningprocess
necessitatestheanalysisoflabeledinputvariablesxand
correspondingoutputvariablesy.Inthesupervisedlearning
trainingphase,theAIneuralnetworkexaminesthetraining
datasetoflabeledxinputdatatolearntoclassifytheinput
dataidealistically.Thehigherthequalityandquantityofsuch
labeleddatasetsthebettertheAIneuralnetwork’slearning
algorithmduringthesupervisedtrainingprocess.Accordingly,
theevolution,improvements,andgrowthofAIiscorrelatedwith
theevolution,improvements,andgrowthinmicrotaskswork.But
alas,microtasksplatformsystemsaresubjecttosignificant
limitationsthatinhibittheevolutionofAI.
```
```
LackofScalingSolutionsinCentralizedMicroTaskMarket
```
# **`Overpricing`**
```
Theexistingcentralizedmarketplacesformicrotaskworkcannot
adequatelyfulfilltheincreasingdemandforhighqualitymicro
taskworkforAIlabeledtrainingdatasets.Firstandforemost,
thecoststructureformicrotaskworkincentralizedsystems
thatnecessitateintermediationresultsinsignificant
overpricingwithoutbenefitingthemicrotaskworkersdirectly.
Thecoststructuresuboptimalitycanbetracedbacktoseveral
factors.Alltoohumanshortcomingsofmicrotaskworks,suchas
```
```
limitedattentionspan,irrationality,andinaccuraciesresult
inverificationrequirementsformicrotaskwork.However,
manualverificationofmicrotaskworkissubjecttothesame
humanlimitations.
```
# **`Duplication of Work`**
```
Inanattempttoensurequalityofresultsandminimizethe
impactofthehumanlimitationsoftheirworkers,requestersin
centralizedmicrotaskstructuressetupteamsofupto15
workerstoperformthesametaskinanefforttoforma
consensus.Themultiplicationofworkinherentinthisprocess
significantlyincreasesthecostofmicrotaskwork.Requiring
requesterstopayproportionally(e.g.upto15times)forwork
perprojectresultsinwaste.Thenecessityofmultiplicationof
workalsosubjectsmicrotaskworkerstolowerratesandlackof
paymentincreases.Moreover,becauseunmanagedcentralizedmicro
taskplatformsdonotsupplyconsumerinterfacesneededto
accomplishspecifictasks,requestersofmicrotaskworkare
forcedtoeitherbuildtheirowntoolsorpaylargefeesto
startupsthathopetocapturetheenterprisemarket.Both
optionsarenecessaryincurrentcentralizedsystemsbutalso
resultinunderutilizationofresources.
```
# **`No Access for Workers`**
```
Mostimportantly,thecirca38%ofthelaborpoolthatis
unbankedbutskilleddoesnotcurrentlyhaveaccesstothe
centralizedmicrotaskmarketplaces.Withoutabankaccount,
workerscannotcontributeandprofitfromtheexisting
centralizedmicrotaskmarketplace.Furthermore,evenforthose
inthelaborpoolwhodohavebankaccounts,micro-taskworkis
oftenassociatedwithpracticalproblemssuchashighfeesof
intermediaryfinancialinstitutions,lostorotherwiseaffected
payments,lostchecks,amongotherissues.Finally,micro
workersincentralizedsystemsarefacedwithinvasive,privacy
challenging,timeconsuming,andunclearsignupandapproval
processesthatcreatemarketentrybarriersformicrotask
workers.
```
# **Gamification to Enhance Micro Task Work**
```
Thegamificationofmicrotaskworkcanhaveseveraleffectson
theprocessofproducinghigh-quality,diversedatasetsforAI
learning.Gamificationreferstotheapplicationofgamedesign
principlesandmechanicstonon-gamecontexts,suchasmicro
taskwork.
```
```
Gamificationcanimpacttheprocessofproducinghigh-quality,
diversedatasetsforAIlearninginthefollowingways:
```
```
1.IncreasedEngagement:Gamificationtechniques,likeadding
points,levels,leaderboards,orrewards,allofwhichmaybe
utilizedexclusivelywithintheplatform,canenhancethe
engagementofcrowdworkers.Byintroducingacompetitiveor
rewardingelement,gamificationmotivatesworkerstoactively
participateandcompletetasksmoreefficiently.Thisincreased
engagementcanleadtohigher-qualityoutputsasworkersare
moreinvestedintheprocess.
```
```
2.QualityControl:Gamificationcanbeutilizedtoimprovedata
qualitythroughmechanismslikeconsensus-basedvotingorpeer
review.Workerscanreviewandrateeachother'scontributions,
earningpointsorrecognitionforaccurateandconsistentwork.
Thisapproachhelpsidentifyandresolvediscrepancies,ensuring
betterqualitycontrolinthedatasetcreationprocess.
```
```
3.SkillDevelopment:Gamificationcanenableskilldevelopment
```
```
amongcrowdworkers.Byprovidingchallengesorlevelsthat
progressivelyincreaseindifficulty,workerscanenhancetheir
annotation,validation,orcleaningskillsovertime.Thisskill
developmentcanresultinimprovedaccuracyandefficiencyin
datasetcreation.
```
```
4.CrowdworkerRetention:Gamificationelementslikebadges,
achievements,orvirtualcurrencies,allofwhichmaybe
utilizedexclusivelywithintheplatform,canenhance
crowdworkerretention.Byrecognizingandrewardingworkersfor
theircontributions,gamificationfostersasenseof
accomplishmentandencouragesthemtocontinueparticipatingin
microtaskwork.Thisretentioniscrucialformaintaininga
consistentpoolofexperiencedandreliableworkersfordataset
creation.
```
```
5.ScalabilityandSpeed:Gamificationcanhelpexpeditethe
datasetcreationprocessbyencouragingworkerstocomplete
tasksmorequickly.Byintroducingtime-basedchallengesor
offeringbonusesfortimelycompletion,gamificationcan
increasethespeedatwhichhigh-qualitydatasetsaregenerated,
enablingscalabilityforlarge-scaleAIprojects.
```
```
It'simportanttonotethatwhilegamificationcanpositively
```
```
impactengagementandproductivity,properdesignand
implementationincombinationwithgamificationlogicof
reputationgovernancearecrucial.Throughthecombinationof
decentralizedgovernanceandgamificationofmicrotaskworkALE
platformintendstoensurethatgamificationtechniquesdonot
compromisethequalityandaccuracyofthedataset.Balancing
```
```
thegamificationelementswithappropriatequalitycontrol
measuresviadecentralizedgovernanceisessentialtomaintain
theintegrityofthedatasetcreationprocess.
```
# **Case Study - Code Review Platform**
```
TheCodeReviewPlatformisoperatedbyanaffiliatedentityand
servedasatestingenvironmentinthepast.Itisaccessible
underthefollowingURL:https://crdao.ossa.dev/
```
## **Community Audit for Code Review**
```
ScalingtheCodeReviewPlatformoperationsasaproofpointis
akeyobjectiveforALE.TheCodeReviewPlatformusesseveral
keyreputationrelatedmetrics,asillustratedbelow,toscale
itsoperationsovertime.
```
```
Toenablethepublictoparticipateinandcontributetothe
CodeReviewPlatform,ALEisusingagamifiedreputationsystem.
AnyusercanaccessandcontributetotheCodeReviewPlatform
andearnreputationscoresontheplatform.Whilethecode
reviewsthemselvesarerelegatedtotheexpertmembersofthe
```
```
CodeReviewPlatform,thepubliccansupporttheultimatecode
reviewsviagamifiedengagements.
```
```
MechanicalTurk(asdefinedabove)functionalityisacore
applicationoftheCodeReviewPlatformProtocolandtheCode
ReviewPlatformecosystem.Otherusecasesandapplicationswill
evolveontheCodeReviewPlatformovertime.TheCodeReview
Platformarchitecturewillevolveinanysettingthatsupports
theethicaladvancementofthecryptoevolution.
```
Growth Potential of the Smart Contract Industry
```
Thesmartcontractindustryhasenormousgrowthpotential.
DifferentmeasureshelpassessthatgrowthincludingtheCAGR
(compoundedannualgrowthrate)usedaswellasthecurrent
valuationofthesmartcontractindustry.Estimatesofthesmart
contractindustry’sfuturevalueby2032rangefrom1to2.5
billiondollars.9
```
> 9 `Varying group estimates are listed below from low to high.Group Estimates for the Smart Contract Industry’s Value by 2032:`
> **`1. SNS Insider: $1 billion, at a CAGR of 24.2%`**
> **`1.1.`** _`Smart Contracts Market Size`_ `, SNS Insider,` <u>`https://www.snsinsider.com/reports/smart-contracts-market-1542`</u> `(last visited Jan. 4, 2024).`
> **`2. Verified Market Research: $1.2 billion, at a CAGR of 26.4%`**
> `2.1.` _`Smart Contracts Market Size and Forecast`_ `, Verified Market Research,` <u>`https://www.verifiedmarketresearch.com/product/smart-contracts-market/`</u> `(last visited Jan. 4, 2024).`
> **`3. Valuates Reports: $1.4 billion, at a CAGR of 24.2%`**
Smart Contract Vulnerabilities Undermine Industry Growth
```
Thegrowthofthesmartcontractindustryacrossindustry
vectorsisaffectedbytheattackvectorspertainingtosmart
contracts.Smartcontractbugscanresultinfinancialloss,
reputationalloss,increasedsmartcontractcosts,legalissues,
and,inextremecases,destructionofthesmartcontract.10
```
- 3.1. _`Global Smart Contracts Market Research Report`_ `, Valuates Reports,` <u>`https://reports.valuates.com/market-reports/QYRE-Auto-31L1599/global-smar t-contracts`</u> `(last visited Jan. 4, 2024).`
**`4. Acumen Research & Consulting: $1.417 billion, at a CAGR of 22.8%`**
- `4.1. Smart Contracts Market Size: Global Industry, Share, Analysis, Trends and Forecast 2023 – 2032, ACUMEN RESEARCH & CONSULTING, https://www.acumenresearchandconsulting.com/smart-contracts-market (last visited Jan. 4, 2024).`
**`5. Future Market Insights: $1.5 billion, at a CAGR of 23.5%`**
- `5.1.` _`Smart Contracts Market Outlook (2022 to 2032)`_ `, Future Market Insights,`
```
https://www.futuremarketinsights.com/reports/smart-contracts-market(last
visitedJan.4,2024).
```
**`6. Allied Market Research: $2.5 billion, at a CAGR of 29.6%`**
- `6.1. Smart Contracts Market Research, 2032, ALLIED MARKET RESEARCH, https://www.alliedmarketresearch.com/smart-contracts-market-A144098 (last visited Jan. 4, 2024).`
> `10 See David Balaban,` _`Navigating The Security Challenges Of Smart Contracts`_ `, Forbes (Feb. 11, 2023, 6:33 AM),`
```
https://www.forbes.com/sites/davidbalaban/2023/02/11/navigating-the-security-ch
allenges-of-smart-contracts/?sh=14006afd4992.MajdSoud,GrischaLiebel&
```
```
MohammadHamdaqa,PrAIoritize:LearningtoPrioritizeSmartContractBugsand
Vulnerabilities(WorkingPaper),https://arxiv.org/pdf/2308.11082.pdf.Sherman
Lee,BlockchainSmartContracts:MoreTroubleThanTheyAreWorth?Forbes(Jul
10,2018,11:38PM),
```
```
https://www.forbes.com/sites/shermanlee/2018/07/10/blockchain-smart-contracts-m
ore-trouble-than-they-are-worth/?sh=74b5654523a6.HaozheZhou,AminMilaniFard
&AdetokunboMakanju,TheStateofEthereumSmartContractsSecurity,2
JournalofCybersecurity&Privacy358(2022).HantingChuetal.,ASurveyon
SmartContractVulnerabilities:DataSources,Detection,andRepair,196
INFORMATION&SOFTWARETECHNOLOGY1(2023),
```
```
https://www.sciencedirect.com/science/article/pii/S0950584923000757.MacKenzie
Sigalos,BugPuts$162MillionupforGrabs,SaysFounderofDeFiPlatform
Compound,MSNBC(Oct.3,2021,2:41PM),
```
```
https://www.cnbc.com/2021/10/03/162-million-up-for-grabs-after-bug-in-defi-prot
```
```
ocol-compound-.html.TamerAbdelaziz&AquinasHobor,SmartLearningtoFind
DumbContracts,
```
```
https://www.usenix.org/system/files/usenixsecurity23-abdelaziz.pdf.Fabio
Grittietal.,ConfusumContractum:ConfusedDeputyVulnerabilitiesin
EthereumSmartContracts,
```
```
Whilenocleardataexiststoestimatehowmuchsmartcontract
```
```
attackvectorswilllimitthegrowthofthesmartcontract
industry,thesmartcontractsindustryisstillestimatedto
growtoseveralbilliondollarsinthenextdecade—despitethe
costofsecurityauditsandfinanciallossesfrombadactors
exploitingsmartcontractsbugs.Estimatesonfinancialloss
attributabletosecuritybreachesandattachvectorsofsmart
contractsrangefromhundredsofmillionstobillionsof
dollars.Forexample,between2016and2018,sevencybersecurity
incidentsoccurredinEthereumsmartcontractsresultingin
financiallossesofover$289million.11In2021alonetotal
financiallossfromsmartcontractbugswasestimatedat$680
million.12Someestimatesputthecurrentglobalfinancialloss
```
```
duetosmartcontractvulnerabilitiesover6billiondollars.13
```
```
https://www.usenix.org/system/files/usenixsecurity23-gritti.pdf.Inonestudy,
researchersfoundthat127high-impactattackswereresponsibleforfinancial
lossestotaling$2.3billion.StefanosChaliasosetal.,SmartContractand
DeFiSecurity:InsightsfromToolEvaluationsandPractitionerSurveys
(WorkingPaper),https://www.doc.ic.ac.uk/~livshits/papers/pdf/icse24.pdf.
11AymanAlkhalifahetal.,AMechanismtoDetectandPrevent
EthereumBlockchainSmartContractReentrancyAttacks,3
FRONTIERSINCOMPUTERSCIENCE1(2021),
```
```
https://www.frontiersin.org/articles/10.3389/fcomp.2021.598780/f
ull.
12ThomasClaburn,SmartContractDevelopersNotReallyFocusedon
Security:WhoKnew?Reporter(Apr.26,2022),
https://www.theregister.com/2022/04/26/smart_contract_losses/.
13StefanosChaliasosetal.,SmartContractandDeFiSecurity:
InsightsfromToolEvaluationsandPractitionerSurveys(Working
Paper),
https://www.doc.ic.ac.uk/~livshits/papers/pdf/icse24.pdf.
```
# Solutions for Smart Contract Vulnerability
```
Adiversesetofproposedsolutionshasemergedtoaddresssmart
```
```
contractvulnerabilitiesissuesandreducetheoverallnumberof
attacks.Onepotentialsolutionistheuseofasmartcontract
compiler.14Forexample,onesuchcompilerisHCC,which
automaticallyinsertssecurityhardeningchecksatthe
source-codelevel.15HCCdevelopsacodepropertygraph(CPG)to
modelcontrol-flowsanddata-flowsofagivensmartcontract.
DuetotheCPGnotation,HCCcanbeappliedtovarioussmart
contractplatformsandprogramminglanguages.HCCdevelopers
havedemonstrateditefficientlymitigatesreentrancyand
integerbugs.16TheyalsoshowhowtointegrateHCCwithinother
blockchainplatformssuchasHyperledgerFabric.Their
```
```
evaluationon10kreal-worldcontractsdemonstratesthatHCCis
highlypractical.17
```
```
Alteringthemethodologyofbugclassificationandvulnerability
```
```
analysisprovidesanotherpromisingapproach.Inonestudy,
researchersproposetwonewvulnerabilityclasses:distributed
```
> `14 Jens-Rene Giesen et al., Practical Mitigation of Smart Contract Bugs (Working Paper),` <u>`https://arxiv.org/pdf/2203.00364.pdf.`</u>
> `15` _`Id`_ `.`
> `16` _`Id`_ `.`
> `17` _`Id`_ `.`
```
systemprotocol(DSP)anddistributedsystemresourcemanagement
(DRM).18
```
# AI-Driven Solutions
```
TheintegrationofblockchaintechnologyandAIhasimmense
promise.19AItechnologyappearsbestsuitedtoeliminatingbugs
throughenhancedbugtrackingandbugaudits.Arecentreviewof
over100researchpapersrevealedthatintegratingthetwo
technologiesresultsincreasesthesecurity,efficiency,and
productivityoftheapplications.20
```
```
Mostbugtrackingisdonemanuallybysoftwareengineers,which
impairsbugtriaging.21Toaddressthisproblem,researchers
proposePrAIoritize;anautomatedapproachforpredictingsmart
```
> `18 Wesley Dingman et al., Defects and Vulnerabilities in Smart Contracts, a Classification Using the NIST Bugs Framework, 73 INTERNATIONAL JOURNAL OF NETWORKED & DISTRIBUTED COMPUTING 121 (2019),`
```
https://www.atlantis-press.com/journals/ijndc/125913574/view?ref
=metastate.
```
> `19 Rashi Saxena, E. Gayathri & Lalitha Surya Kumari, Semantic Analysis of Blockchain Intelligence with Proposed Agenda for Future Issues, 14 International Journal of System Assurance Engineering & Management 34 (2023),`
```
https://link.springer.com/article/10.1007/s13198-023-01862-y.
20Id.
21MajdSoud,GrischaLiebel&MohammadHamdaqa,PrAIoritize:
LearningtoPrioritizeSmartContractBugsandVulnerabilities
(WorkingPaper),https://arxiv.org/pdf/2308.11082.pdf.
```
```
contractbugprioritiesthatassistsoftwareengineersin
```
```
prioritizinghighlyurgentbugreports.22
```
```
Enhancedsmartcontractauditingcanbeaccomplishedthrough
deeplearningtechniques.Inarecentstudy,researcherstrained
threedeepmodelsfordetectingvulnerabilitiesinsmart
contract:Optimized-CodeBERT,Optimized-LSTM,and
Optimized-CNN.23Experimentalresultsshowthat
Optimized-CodeBERTmodelsurpassesothermethods,achievingan
f1-scoreof93.53%.24Topreciselyextractvulnerability
features,theyacquiredsegmentsofvulnerablecodefunctionsto
retaincriticalvulnerabilityfeatures.UsingtheCodeBERT
pre-trainingmodelfordatapreprocessing,theauthorscould
capturethesyntaxandsemanticsofthecodemoreaccurately.
Theauthorsevaluateditsperformanceusingthe
SolidiFI-benchmarkdataset,whichconsistsof9369vulnerable
contractsinjectedwithvulnerabilitiesfromsevendifferent
types.
```
> `22 Xueyan Tang, Yuying Du, Alan Lai, Ze Zhang & Lingzhi Shi, Deep Learning ‑ based Solution for Smart Contract Vulnerabilities Detection, 13 SCIENTIFIC REPORTS 1 (2023),` <u>`https://www.nature.com/articles/s41598-023-47219-0.`</u>
> `23` _`Id`_ `.`
> `24` _`Id.`_
```
Inanotherstudy,researcherstrainedartificialneuralnetworks
(ANN),long-shorttermmemory(LSTM),andgatedrecurrentunit
models(GRU)andcomparedtheiraccuracy,precision,recall,and
receiveroperatingcharacteristic(ROC)curvevalues.25The
networkwastrainedonanopenGoogleBigQuerydatasetwith7000
samples.TheirresultsdemonstratedthattheLSTMmodel
outperformsANNandGRU.26Lastly,AItechnologyhasthe
```
```
potentialtogenerallyimprovesmartcontractsecurity.27
```
# Market for Code Reviews
```
Themarketforcodereviewsin2024isdominatedbycentralized
marketparticipants.Theexistingcodereviewindustryis
```
```
subjecttosignificantcost,inefficiencies,barrierstoentry,
andlackofassurancesforcodereviewjobposters,amongother
lackofclientservicesforcodereviews.In2024,thecode
reviewindustryisdominatedbyseveralkeyplayerswhocan
chargeexorbitantandmonopoly-likeprices.Despitethesehigh
prices,thecodereviewprocessissubjecttosignificantflaws.
```
> `25 Rajesh Gupta et al., Deep Learning-based Malicious Smart Contract Detection Scheme for Internet of Things Environment, 97 COMPUTERS & ELECTRICAL ENGINEERING 1 (2022),` <u>`https://www.sciencedirect.com/science/article/pii/S0045790621005 19X.`</u>
> `26` _`Id.`_
> `27 Moez Krichen,` _`Strengthening the Security of Smart Contracts through the Power of Artificial Intelligence`_ `, 12 Computers 107 (2023),` <u>`https://www.mdpi.com/2073-431X/12/5/107.`</u>
```
Firstandforemost,theselectionprocessforthereviewerisan
```
```
ongoingchallengefortheexistingcodereviewprocessinlegacy
codereviewfirms.Themorehierarchicalthecodereview
processis,theloweristhequalityofthereviewedcode.It
makesintuitivesensethatthemoredevelopersreviewagiven
codeset,thehigherthecodequalitymayturnouttobe.
However,inthelegacyreviewprocess,thefirstreviewerwithin
thehierarchicalstructureofthecodereviewprocessoftengets
thehighestpriorityandisoftenmerelyfollowedwithminor
upgradesbyfollow-onreviewers.Thecollectiveofreviewersis
alsonotincentivizedtofindflawsinthecodetooptimizecode
asaworkproductofthecollective.Rather,itisoftenseenas
theworkproductoftheinitialreviewerwithminorinputfrom
follow-upreviewers.Moreover,themorepeoplereviewthecode
withcommentsthataskforclarification,themorelikelyit
becomesthatthecodebecomessimplerandclearer,whichinturn
typicallyincreasescodequality.However,thatisnotpossible
inhierarchicalreviewprocesses.Thehierarchicalapproachto
codereviewsundermineslong-termparticipationwithopinions
fromtheedgesofthereviewerspectrumbecausethosereviewers
eitherhavenoaccessorareinnopositiontohelpreviewthe
code.Inotherwords,themorehierarchicalthecodereview
```
```
processandthemorebarrierstoentry,thelowerthequalityof
thecode.
```
# Single Points of Failure
```
Intheexistinglegacycodereviewprocess,theviewsofthe
reviewerandtheintentofthecodeauthorareoftenatodds
witheachotherwithoutanycrowdcontrol.Becausethecode
reviewermaywishtoimposetheirlogiconthecodeauthor,the
codeauthormayberequiredtorewritecodeoverandovereven
thoughthecorefunctionalityofthecodeissoundanddangerous
issueswerecontrolledfor.Thiscanbehighlytime-consuming
andinefficient.Italsocallstheoverallroleofthecode
reviewprocessintoquestion.Instead,acodereviewshould
focusonthefunctionalityofthecodeandonkeepingmistaken,
badlyconstructed,anddangerouscodeout.
```
```
Ifthesingleauthorofacodereviewhasmissedsomethingand
thefollow-onreviewerfocusedentirelyonthefirstreviewer's
concerns,thecodereviewhasahigherriskoflackofaccuracy.
Crowdwisdomisonewaytocorrectpossiblemyopiaandsingle
pointsoffailurethroughstandardlegacycodereview.
```
Timing
```
Dependingonthesettingofthecodereview,codereviewsin
legacysystemscanlastweeksandmonths.Thiscanbe
exacerbatedbymarketconditionsinthedigitalassetmarket.
Thesesignificantdelayscanimpactdevelopmentandmayrequire
completerewritingofcontractsbecausetheunderlyingprotocol
mayhaveupgradedcorelibrariesduringthecodereview.
```
Current Players in the Field
```
Eventhoughmostcodereviewfirmshelpclientswhohopeto
decentralizedifferentpartsoftheindustryandcapitalizeon
efficienciescreatedbydecentralizinglegacysystems,thecode
reviewindustryismainlydominatedbyafewplayers.
```
```
Thecurrentmarketdynamicsdominatedbythetop5auditfirms
alsocreatehighbarrierstoentryfornewplayerstoenterinto
thecodereviewmarket.
```
```
Giventhesedownsidesintheexistingcodereviewmarket,itis
```
```
abitironicthatoneofthestrongestformsofexploitationand
centralizedeconomiesofscalearebeingcreatedinamarket
thatis,fromtheoutset,supposedtohelpsupportthe
decentralizationofdisparateindustries.
```
# Overpricing
```
Asaresultofthecentralizationoftheindustry,mostcode
reviewsaresignificantlyoverpriced.Clientsprettymuchpay
anypricetogetthestampofapprovalfromoneofthetop5
auditfirms.
```
# No Controls
```
Thecentralizedpoweralsounderminesattemptsbyotherindustry
playerstocreateinternalorexternalcontrolsonthequality
ofcodereviews.Asaresult,thepublichasnoorveryweak
controloverthequalityofcodereviewservicesitreceives.
jobposterscannotaffordtolookforbetter-pricedcodereviews
andareforcedintoconsiderablepricingtoobtainmarket
acceptanceoftheirproducts.Inturn,thecentralizationof
```
```
themarketunderminesanyformofdownwardpricepressure.
```
```
Becauseofthepowerofthelimitedplayersovertheoverall
marketandtheprocessofthecodereviewanditsoutputs,the
qualityofcodereviewisoftensuboptimal.
```
```
Moreover,thereislittleornorecourseforclientsincasesin
whichthecodeprovedtobeflawedevenafterfunctionalityand
qualityreview.
```
# **Code Review DAO**
```
TheCodeReviewDAO(CRDAO)istacklingmanyoftheissuesthat
afflictthemoderncodereviewmarket.
```
```
TheCRDAOprovidesanewcodereviewplatform,whichfacilitates
```
```
adecentralizedcommunity-drivencodereviewprocessthat
utilizesabiddingprocessoncodereviewstodrivepricesdown
(the“CodeReviewPlatform”).Itprovidesopenaccessforcode
reviewsfromanyonewhoqualifies–notjustmembersofthefew
codereviewfirms.Atthesametime,itprovidesfull
incentivizationforcommunitycodereviewsthroughits
decentralizedgovernanceframework.
```
```
Givenitsuniversalaccessandpricediscoverymethodology
(throughapublicbiddingprocess),theCodeReviewPlatform
createslowbarrierstoentryinthecodereviewmarket.Anyone
canjointheCodeReviewPlatformbysubmittinghigh-quality
codereviewsthroughtheCodeReviewPlatformportal.
```
```
TheCodeReviewPlatformalsousesacommunitypolicingand
auditmethodologyforcodereviews.ThisisenabledbytheCode
```
```
ReviewDAO28(CRDAO)governancemodel.Moreover,theCRDAO
governanceandpolicingfunctionsensurelessduplicationof
codereviews.CodeReviewPlatformmaintainsstrongincentives
forcommunity-drivencodereviewaudits.
```
```
TheCodeReviewPlatformcanfunctionasafirst-roundcode
reviewormulti-roundcodereviewwithdifferentcodereview
teams.Insummary,jobpostersreceivelow-priced,high-quality
codereviewswithcommunityvalidationofreviews.
```
Feedback Loops
```
CodeReviewPlatformprovidesanearlyfeedbacklooponcode
reviewsforthedevelopercommunityatamuchlowerpricethan
thetraditionalcodereviewandauditmarket.
```
```
TheCRDAOcrowdcontrolsfilteroutidiosyncraticcodereviewer
preferences.Codereviewsinlegacycodereviewfirmsareoften
highlysubjectivewhichleadstorathersuboptimaloutcomes
withoutcrowdcontrols.Nosingledevelopermayagreeonagiven
setofcodeanditsintendedfunctionalityandqualityin
achievingthecodedobjectives.Thiscanbeattributedto
```
> `28 DAOs are short for Decentralized Autonomous Organizations.` <u>`https://en.wikipedia.org/wiki/Decentralized_autonomous_organization#:~:text=Dec entralized%20autonomous%20organizations%20are%20typified,dissemination%20of%20a`</u>
> <u>`%20distributed%20database.`</u>
```
differentprogramminglanguageswithdifferentstylesandunique
andoftenidiosyncraticpreferences.Instead,theALEmandates
thatreviewsaresubjecttocrowdreviewandpolicingvotesby
theCRDAOcollective.Accordingly,codereviewersareless
likelytoengageinhighlyidiosyncraticreviewsastheywould
needtofearslashingofreptokenscoresandlossofstanding
inthecommunity.
```
# Price Discovery
```
Theexistingcodereviewmarketdoesnotprovidepublicly
transparentpricingofcodereviewservices.Assuch,the
existingmarketarguablyharmsthepublicforthebenefitofthe
fewmarketplayersanditsclients.Intheexistingsystem,the
unilateralpricingisnotdisclosedbecauseboththeclientand
codereviewermaynotbenefitfrompublicscrutinyofthe
prices.
```
```
TheCodeReviewPlatformusesauniqueandfullytransparent
pricediscoverymechanismforcodereviews.TheCodeReview
Platform’spricediscoverymechanismworksasfollows:
```
- `Job poster posts the job on the Code Review Platform portal`
- `DAO internal - Job price discovery`
- `Internal DAO bids are collected in a table format next to the job posting terms and show the bids up and down on the terms 1. Time +- 2. Price - + 3. Reputation score range of the bidder (actual reputation of bidder is within the range), 4. Reputation stake of the bidder`
- `At the end of the internal bidding period - the job poster reviews all the bids and selects the winner and prices are revealed`
- `If the job poster does not select a winner`
- `after a grace period, it automatically goes to public bidding and prices are never revealed.`
- `Bidding is not public during the bidding process and (fully anonymized through reputation score ranges)`
- ■ `All bids on the post and winner will become public at the end of the selection when the job poster has picked the winner`
- `Public - Job price discovery`
○ `If no bids for` _`n`_ `days internally in the CRDAO community member pool, the job post will be made public automatically for public bidding - public bidding follows these parameters.`
- `External bidder posts DoS`<sup>`29`</sup> `fee to get onboarded(admin approval)`
■ `External bids are collected in a table format next to the job posting terms and show the bids up and down on the terms 1. Time +- 2. Price - +`
■ `At the end of the external bidding period - the job poster reviews all the bids and selects the winner`
- `If the job poster does not select a winner after a grace period, it automatically ends the bidding process with no winner.`
- `Bidding is not public during the bidding process and (anonymized if preferred by bidder)`
- ■ `All bids on the post and winner will become public at the end of the selection when the job poster has picked the winner`
> `29 Denial of Service: https://en.wikipedia.org/wiki/Denial-of-service_attack`
● `If the job poster accepts a bid (DAO internal or public) - the job poster now has to post a deposit for the value of the agreed code review job into a smart contract.`
```
Thispricediscoverymechanismservesakeypublicservice
functioninthatitenablesfull-pricetransparencyfor
consumersbasedonthevisibilityoftheinternalandexternal
bidsonjobposts.Thispricetransparencyisuniqueand
unprecedentedinamarket.
```
```
Pricediscoveryisakeypublicservicesfunctionbecause,
withoutpublicpricing,consumerscannotrealisticallyselect
theserviceproviderthatprovidesthehighestvaluetothejob
poster.Thelackoftransparencyenablesinsiderdealstothe
detrimentoftheclients,whoareforcedintothepriceagroup
offirmsdictates.
```
# Standards
```
TheCodeReviewPlatformalsofillsthevoidleftinlegacy
reviewswithoutcommonstandards.TheCodeReviewPlatform
createsacompendiumofreviewsandthroughitacommonstandard
forcodereviewsthatareotherwiselackinginlegacycode
```
```
reviewenvironments.Shouldacollectivereviewcodeundera
commonsetofstandards,thestandardshelpguideboththe
reviewersandthecollectivetocometoacommonformof
expectationsontheappliedfunctionalityandqualityoutcomes
forthecode.
```
# Speed
```
ThefeedbackprovidedbytheALEforthedevelopercommunity
enablesrisk-takingfordevteamswhowishtomovequickly
throughtheirgovernanceandupgradeprocess,whichinturn
enablesacceleratedgrowthandscalingofexperimentation.
```
# Code Testing
```
ThecodereviewsprovidedbytheCodeReviewPlatformprovidea
```
```
firstinstantiationofflathierarchy-drivendecentralized
peer-reviewedcodereviews.AlltestingperformedbytheCode
ReviewPlatformfollowsthecorestandardsestablishedforthe
CRDAOcommunity.Alltestingandstandardsaresubjectto
constantreviewandexperimentationandarecontinually,
dynamically,andevolutionarilyupdatedinaconstantfeedback
```
```
loopbetweenallconstituents,thatisbetweentheCRDAOmember,
```
```
thejobposters,andpublicbiddersandapplicantsforCRDAO
```
```
communitymemberstatus.
```
# Community Audit
```
Theauditstartswiththecommunitydiscussionofthecode
review,followedbyaninformalvoteonthecode.Thecommunity
thatbidsforthepostedjobsandthenreviewsthesubmittedjob
andvotesonitaftertheforumdiscussionisconstitutedbythe
evolvingcommunitymembershipoftheCodeReviewPlatform.The
informalvoteshowsallCRDAOmembersthecollectivewisdomas
appliedtotheworkproductexaminedinthecodereview.Once
theentirecommunityknowshoweachmemberfeelsaboutthecode
reviewexamined,theCRDAOnowvotesinaformalvoteinwhich
thereputationtokens30stakedareatrisk.Thissequenceof
votesprovidesjobposterswithsignificantassurancesthatthe
codeexaminedandtheCodeReviewPlatformreportonthecode
adherestothehigheststandardsofqualityavailable.
```
> `30 “Reputation Tokens” are non-transferable tokens that cannot be valued and represent the reputation a member has within the community. They simply mirror a scoreboard. Each member of the CRDAO holds such reputation tokens to measure the merit and quality of their input into the community, presenting the member's unique and individual reputation status according to certain criteria established by the CRDAO. At the time of this writing, the criteria for reputation token allocation to CRDAO members revolved around the respective members’ ability to contribute to the CRDAO code reviews, either as a job performer or as part of the community audit.`
Code Review Platform Code Review Process
```
ThecodereviewprocessoftheCodeReviewPlatformrevolves
aroundCRDAOcommunityengagementwhichminimizesissuesoflack
ofcrowdcontrols,lowerstimerequirementsforcodereviews,
lowerspricesofcodereviews,increasesdeveloper
```
```
participation,andincreasesoverallfeedback.
```
# **AI Learning Ecosystem Platform**
## **Scaling the Code Review Platform with Micro Task Work**
```
ViatheCodeReviewPlatformasaproofpoint,ALEistapping
intothisrisingmicrotaskmarketforAItrainingdataby
providinguserswithgamifiedaccesstocodereviews.Key
examplesthatsetprecedentforthegamificationincludeAxie
InfinityandotherscalingEthereumgamesthatattained
worldwideaudiences.
```
## **Removing Cost of Micro Task Work Duplication**
```
Centralizedmicrotaskworkrequiresduplicationtoensure
quality.Alltoohumanshortcomingsofmicrotaskworks,suchas
limitedattentionspan,irrationality,andinaccuraciesresult
```
```
inverificationrequirementsformicrotaskwork.However,
manualverificationofmicrotaskworkissubjecttothesame
humanlimitations.Inanattempttoensurequalityofresults
andminimizetheimpactofthehumanlimitationsoftheir
workers,requestersincentralizedmicrotaskstructuressetup
teamsofupto15workerstoperformthesametaskinaneffort
toformaconsensus.31Themultiplicationofworkinherentin
thisprocesssignificantlyincreasesthecostofmicrotask
work.Requiringrequesterstopayproportionally(e.g.upto15
times)forworkperprojectresultsinwaste.Thenecessityof
multiplicationofworkalsosubjectsmicrotaskworkerstolower
ratesandlackofpaymentincreases.
```
```
Bycontrast,intheALEplatform,thecommunityorganization
softwareenablesareadilyavailableindicatorofhowreliablea
workerorrequesterisontheALEPlatform.Themicrotask
worker’sreputationscoreisameasureoftheworker’shistory
ofcompletingmicrotasksontheALEPlatformqualitatively
accurate,efficient,andconsistent.Therequester’sreputation
scoreisameasureoftherequester’shistoryofinteracting
withmicrotaskworkersontheALEnetwork.TheALEreputation
```
> `31 Neeraj Kumar,` _`Effective Use of Amazon Mechanical Turk (MTurk); Tips and techniques for better usage of Amazon Mechanical Turk for researchers,`_ `NEERAJ KUMAR (May 2013, updated May 8, 2014)` _`,`_ `http://neerajkumar.org/writings/mturk/;`
> `Rory O’Reilly,` _`How the Gems Protocol Reduces Consensus by Redundancy`_ `, GEMS (Nov. 27, 2017),`
> `https://blog.gems.org/how-the-gems-protocol-reduces-consensus-by-redundancyb151de80ecb8.`
```
scoreisformedandlinkedtotherespectivenetwork
participant’swalletaddress.
```
```
Marketfactorsbalancetheequilibriumofsupplyanddemandof
microworkontheALEPlatformbasedontheworkersand
requestersreputationscores.Ifrequestershavealower
reputationscore,workersbecomelesslikelytoaccept
requesters’offers.Inturn,lowreputationscoresformicro
taskworkersresultinalowerlikelihoodofretentionformicro
taskworkontheALEPlatform.Requesterscanselectworkers
basedontheirreputationscore,givingworkersanincentiveto
keepthereputationscoreshighbyperformingmicrotaskswith
highaccuracyandefficiency.Thereputationscoremechanismand
thebuildingofreputationontheALEPlatformallowsworkersto
graduatetotheprivilegeofbeingaverifier.Thereputation
scoremechanismhelpsdiscernmaliciousactorsandsimple
mistakes.Italsoprotectsworkersandverifiersfromfraudulent
requestersandsuboptimallydesignedrequests.
```
```
Insummary,themarketfactorsandmarketdynamicsrelatedto
ALEPlatformreputationscoresenablealoweringofthecostof
duplicationascomparedtocentralizedmicrotaskwork.Ifa
reliableandhighreputationscoreworkercompletesthetasks,
theduplicationmaybebroughtfrom15to5orlessinthe
```
```
decentralizedcodereviewsetup.Thisenablesunprecedented
```
```
scalingofmicrotaskwork.
```
# **Transaction Cost Minimization of Micro Task Work**
```
TheALEPlatformremovestransactioncostsassociatedwithmicro
taskwork.Unlikecentralizedmechanicalturkplatformsthat
requireanexistingbankingrelationshiptoreceiveaccount
transfersforotherwiseunbankedmicrotaskworkers,theALE
Platformoperatesentirelythroughcryptotransactions.
```
# **Gamification of Micro Task Work**
```
Thegamificationofmicrotaskworkcanhaveseveraleffectson
theprocessofproducinghigh-quality,diversedatasetsforAI
learningviatheALEPlatform.
```
```
1.IncreasedEngagement:GamificationtechniquesoftheALE
Platformincludeapointsystem,levels,leaderboards,and
rewards,allofwhichmaybeutilizedexclusivelywithinthe
platform,thatareallocatedviareputationscoresinthe
community.ThissystemofreputationscoringintheALEPlatform
```
```
enhancestheengagementofcrowdworkersintheALEPlatform.By
introducingthereputationscoresasacompetitiveandrewarding
element,theALEPlatoform’sgamificationmotivatesworkersto
```
```
activelyparticipateandcompletetasksmoreefficientlyand
moreresponsively.Thisincreasedengagementcanleadto
higher-qualityoutputsasworkersaremoreinvestedinthe
process.Workersgainreputationscoresandarerewardedwitha
partofanyincomingcompensation(stablecoinsorothermajor
cryptocurrenciespaidbyjobposters)proratatotheir
reputationscores.Thisincentivizesthemtocareaboutthe
reputationscoreandengagewithmorerigorandduediligencein
themicrotaskwork.Iftheydon’tengagewiththecare
required,theysacrificetheirspotintherankingsof
reputationscoreswhichaffectstheirparticipationinthejob
feedistribution,whichispaidoutproratatotherespective
reputationscores.
```
```
2.QualityControl:GamificationintheALEPlatformimproves
dataqualitythroughtheconsensus-basedvotingonworkproducts
ofindividuals.WorkersintheALEPlatformreviewandrateeach
other'scontributions,earningreputationpointsforaccurate
andconsistentwork.Thisapproachhelpsidentifyandresolve
discrepancies,buildconsensus,andensuresbetterquality
controlinthedatasetcreationprocess.
```
```
3.SkillDevelopment:GamificationintheALEPlatformis
focusedonexpertisebuildingandthusenablesskilldevelopment
amongcrowdworkers.Byprovidingchallengesofworksetsin
levelsthatprogressivelyincreaseindifficulty,workerscan
enhancetheirannotation,validation,andcleaningskillsover
time.Allskilldevelopmentisdirectlytraceablethroughthe
reputationscoreofeachworker.Thisskilldevelopmentcan
resultinimprovedaccuracyandefficiencyindatasetcreation.
```
```
4.CrowdworkerRetention:GamificationelementsontheALE
Platformlikebadges,achievementawards,andvirtualcurrency
paymentsthatareallrelatedtothereputationscoreofeach
workercanenhancecrowdworkerretention.Byrecognizingand
rewardingworkersontheALEPlatformfortheircontributions,
gamificationfostersasenseofaccomplishmentandencourages
workerstocontinueparticipatinginmicrotaskwork.This
retentioniscrucialformaintainingaconsistentpoolof
experiencedandreliableworkersfordatasetcreation.Attrition
ratescanfallthroughthegamificationdesigninherentinthe
ALEPlatform.
```
```
5.ScalabilityandSpeed:GamificationontheCodeReview
Platformcanhelpexpeditethedatasetcreationprocessby
encouragingworkerstocompletetasksmorequickly.Thisisdone
inanefforttomaintaintheirreputationscores.Byintroducing
time-basedchallengesinprovidingtheworkandbyoffering
bonusesfortimelycompletion,gamificationCodeReview
Platformcanincreasethespeedatwhichhigh-qualitydatasets
aregenerated,enablingscalabilityforlarge-scaleAIprojects.
```
```
TheALEPlatformemphasizesthedesignandimplementationof
itsprovendecentralizedgovernancelogicincombinationwith
thegamificationlogicinherentinitsreputationgovernance.
Throughthecombinationofdecentralizedgovernanceand
gamificationofmicrotaskwork,ALEPlatformensuresthat
gamificationtechniquesdonotcompromisethequalityand
accuracyofthedataset.Balancingthegamificationelements
withappropriatequalitycontrolmeasuresviadecentralized
governanceisessentialtomaintaintheintegrityofthedataset
creationprocess.
```