1,721,060 research outputs found
VALUE PREDICTION USING K NEAREST NEIGHBOURS ALGORITHM
Get PDFV diplomskem delu smo predstavili uporabo algoritma k najbližjih sosedov za napovedovanje vrednosti predmetov glede na njihove lastnosti. Implementirali smo testni program in preučili delovanje algoritma na podatkih iz resničnega sveta. S pomočjo testnega programa smo ocenili učinkovitost algoritma. Predlagali smo še nekaj možnih izboljšav za naš program.In this diploma work we described the use of k nearest neighbours algorithm for predicting value of objects, depending on their properties. We implemented a test program, and studied how the algorithm worked on real-world data. We evaluated the algorithms efficiency using our test program. We also suggested a couple of possible improvements for our program
The impact of quantization on the performance of deep neural networks
Get PDFV magistrskem delu predstavimo različne tehnike kvantizacije globokih nevronskih mrež in raziskujemo njihov vpliv na učinkovitost modelov.
Na klasifikacijskem modelu ResNet-18 in regresijskem LSTM primerjamo metode kvantizacije med učenjem in po učenju, pri čemer eksperimentiramo s kvantizacijo uteži, aktivacij in gradientov pri bitni širini 16 in 8.
Rezultati so v skladu s pričakovanji in kažejo, da določene metode znatno zmanjšajo velikost modelov in povečajo hitrost sklepanja ob ohranjanju primerljive točnosti, kar omogoča učinkovito implementacijo modelov na napravah z omejenimi računalniškimi viri.This master\u27s thesis presents various quantization techniques for deep neural networks and investigates their impact on model performance.
We evaluate the ResNet-18 classification model and an LSTM regression model using quantization methods applied during and after training, experimenting with the quantization of weights, activations, and gradients at bit widths of 16 and 8.
The results align with expectations and show that certain methods significantly reduce model size and improve inference speed, while maintaining comparable accuracy, thereby enabling efficient deployment of models on devices with limited computational resources
Overfitting prevention in training of multilayer neural networks
Get PDFDiplomsko delo obravnava tematiko strojnega učenja s pomočjo uporabe umetnih nevron-
skih mrež. Te so po svojih sposobnostih in načinu delovanja zelo podobne delovanju
človeških možganov. Imajo sposobnost akumuliranja znanja s tako imenovanim postop-
kom ”učenja”, hkrati pa so sposobne to znanje tudi shranjevati.
Pravilnost delovanja mrež se s postopkom učenja, ki se ponavlja iterativno, povečuje.
Ena izmed glavnih težav pri učenju nevronskih mrež je pojav prekomernega prileganja,
ki se kaže v tem, da mreža ne posplošuje dobro iz učne na testno množico vzorcev. Za
preprečevanje tega pojava je bilo razvitih več tehnik, katerih uporaba, učinkovitost in
primerjava je predmet pričujočega diplomskega dela.This thesis deals with the subject of machine learning by using artificial neural networks.
They are very similar to the human brain in their abilities and way of functioning. They
have the capacity to accumulate knowledge through the so-called “learning” process, but
they are also able to store this knowledge.
The accuracy of artificial neural networks is increased in the process of learning,
which is repeated iteratively. One of the main problems in this process is the emergence
of overfitting. This is because the network does not generalize well from the learning
to the test set. To prevent this phenomenon several different techniques have been
developed, the application and effectiveness of which have been analyzed and compared
in the present thesis
Global illumination using voxel cone tracing
Get PDFGlobalna osvetlitev v računalniški grafiki igra zelo pomembno vlogo pri realizmu upodobljenih slik. Računsko je zelo zahtevna, vendar si želimo globalno osvetlitev v aplikacijah, kot so simulacije in računalniške igre, doseči v realnem času. V tem diplomskem delu si bomo ogledali algoritem vokselskega sledenja stožcu. Ta doseže interaktivno globalno osvetlitev z aproksimiranjem vpadnih svetlobnih žarkov s pomočjo stožcev in z aproksimiranjem scene s pomočjo hierarhične strukture vokslov. Najprej opišemo, kako algoritem v osnovi deluje. Nato predstavimo eno od možnih implementacij algoritma in na koncu prikažemo rezultate naše implementacije.Global illumination in computer graphics plays a very important role in the realism of rendered images. Because of its nature it is very demanding, though we want to achieve global illumination in real-time for applications like simulations and computer games. In this thesis, we will research the algorithm Voxel Cone Tracing, which is achieving interactive global illumination by approximating light rays using cones and by approximating the scene using a hierarchic voxel structure. First, we examine Voxel Cone Tracing in theory. After that, we review our implementation and show the results we get
Using a neural network to control a simulated autonomous vehicle
Get PDFV magistrskem delu smo preučili nevronske mreže in njihovo uporabo za učenje vožnje simuliranih avtonomnih vozil. Pripravili smo 3D-okolje s cesto, po kateri so se vozila učila voziti. Vsako vozilo ima svojo nevronsko mrežo, ki določa hitrost in smer vozila. Preučili in primerjali smo različne pristope učenja – evolucijski pristop in nadzorovano učenje z metodo vzvratnega prenosa napake. Cilj magistrskega dela je bil ustvariti simulirano avtonomno vozilo, ki je sposobno pravilne vožnje po desni strani ceste in se zna izmikati oviram med vožnjo.In the present research, we studied neural networks and their level of application in learning how to drive simulated autonomous vehicles. We created a 3D environment including a road on which the vehicles can train how to drive. Each vehicle has its own neural network that determines the direction and speed of the vehicle. We compared and studied different approaches to learning – evolutionary approach and supervised learning using the backpropagation method. The aim of this master thesis was to create a vehicle that is able to drive on the right side of the road and avoid the obstacles
Machine learning of computer player in Havannah game
Get PDFV zadnjih letih je bil na področju umetne inteligence z uporabo okrepitvenega učenja nevronskih mrež dosežen preboj pri sposobnostih računalnika za igranje iger na deski, kot je Go, pri katerih je bil človek doslej močnejši nasprotnik. V diplomskem delu raziščemo algoritem igranja iger AlphaZero, ki kombinira tehnike preiskovanja dreves Monte Carlo in okrepitvenega učenja nevronskih mrež. Algoritem začne brez posebnega predznanja o dobrih strategijah, vendar se moč algoritma s postopkom učenja, ki se ponavlja iterativno, konstantno povečuje.
V diplomskem delu opišemo in implementiramo osnovno obliko AlphaZero za igranje igre Havannah. Naučimo več različic modela nevronskih mrež, kjer vsak naslednik premaga svojega prednika in postane prvak. S tem pokažemo, da se lahko računalniški igralec uči igranja igre Havannah samo s podanimi pravili igre, tako da je sposoben premagati povprečnega človeškega igralca.In recent years, in the field of artificial intelligence, the reinforcement learning of neural networks has been used to achieve a breakthrough in the ability of the computer players to play board games, such as Go, in which human has been a stronger opponent. In this thesis, we explore the AlphaZero algorithm, which combines Monte Carlo tree search and reinforced neural network learning. The algorithm starts without any special prior knowledge of good strategies, but the algorithm becomes stronger with a learning process that repeats iteratively.
In this thesis, we implement the basic form of AlphaZero for playing the Havannah game. Several versions of the neural network model are trained to play the game, where each successor defeats its predecessor and becomes the champion, thus showing that a computer player can learn to play the Havannah game and win against a human player, simply by being given the rules of the game and not possessing any special prior knowledge of good strategies
Comparing adversarial example attacks on deep neural networks and defensive approaches
Get PDFGloboke nevronske mreže imajo ranljivosti, kot so nasprotniški primeri - neopazne namerne popačitve vhodnih podatkov, ki povzročijo neželeno spremembo izhoda. Ker so nasprotniški primeri prenosljivi, lahko popačitev tvorimo v scenariju črne škatle, brez da bi poznali strukturo ali uteži napadene mreže. V našem delu primerjamo več pristopov k napadu in zaščiti. Robustnost modela ovrednotimo glede na prepričanost v napačno klasifikacijo ter glede na statistično porazdelitev amplitud nasprotniških popačitev. Rezultati kažejo, da so amplitude uspešnih napadov tipično za en velikostni razred višje, če je napad izveden po scenariju črne škatle. Robustnost modela je odvisna od klasifikacijskega problema, arhitekture mreže ter pristopa k zaščiti. Za najbolj učinkovita napada sta se v scenariju bele škatle izkazala napada BIM in MI-FGSM, v scenariju črne škatle pa napada MI-FGSM in FGSM. Najučinkovitejša pristopa k zaščiti sta bila diskretizacija ter virtualno nasprotniško učenje. Pokazali smo tudi, da moramo za verodostojen preizkus učinkovitosti nasprotniškega učenja uporabiti napad, ki ni bil uporabljen v procesu učenja.Deep neural networks suffer from unique vulnerabilities, such as adversarial examples - unobservable perturbations of input data that cause an unwanted change of the output. Adversarial examples are transferrable, which allows us to form an adversarial perturbation in the black-box scenario, without knowing the structure or weights of the attacked network. In our work, we compare various attacks and defensive approaches. We evaluate model robustness based on its confidence in the incorrect classification and the statistical distribution of adversarial perturbation amplitudes. Experiments show an order of magnitude increase in perturbation amplitudes when the black-box scenario is used. Model robustness varies depending on the classification problem, network architecture and defensive approach. According to our results, BIM and MI-FGSM are the most effective attacks in the white-box scenario, and MI-FGSM and FGSM in the black-box scenario. The most effective defenses were discretization and virtual adversarial training. Additionally, we have shown that a valid test of adversarial training requires the use of an attack that was not used during training
NEUROEVOLUTION METHODS
Get PDFPri klasičnem razvoju nevronskih mrež za uporabo na različnih področjih umetne inteligence se pogosto srečujemo s problemom določanja optimalne topologije nevronske mreže, ki ima velik vpliv na njeno kvaliteto. V tem magistrskem delu smo se ukvarjali s problemom avtomatiziranega razvoja topologije nevronske mreže in njenega učenja z nevroevolucijskimi metodami. Področje, ki se ukvarja z razvojem nevronskih mrež s pomočjo evolucijskega algoritma, je nevroevolucija.
Preučili in implementirali smo nevroevolucijske metode NEAT, HyperNEAT in ES-HyperNEAT. Uspešnost metod smo preizkusili na eksperimentu z agenti. Agent se nahaja v okolju in poskuša pobrati čim več kosov hrane ter se izogniti sovražnikom oziroma zidovom.
Nevroevolucijske metode smo primerjali z metodo Q-učenje, ki za učenje nevronske mreže uporablja klasično metodo vzvratnega prenosa napake. Primerjali smo doseženo oceno in časovno zahtevnost. Rezultati so pokazali, da je najuspešnejša metoda HyperNEAT, sledita pa ji ES-HyperNEAT in NEAT. Metoda Q-učenje se je izkazala za najmanj uspešno, saj je glede na nevroevolucijske metode v podrejenem položaju tako po doseženi kriterijski oceni kot tudi po časovni zahtevnosti.In classical development of neural networks for use in different areas of artificial intelligence we are often faced with the problem of optimal neural network topology selection, which has strong impact on neural network quality. In this master work we were studying with problem of automated development and training neural network topologies using neuroevolution methods. Development of NN topology using evolutionary algorithm is called nevroevolution.
We have studied and implemented neuroevolution methods NEAT, HyperNEAT, and ES-HyperNEAT. The methods were compared using an experiment, where an agent tries to collect food and avoid enemies and walls.
Neuroevolutionary methods were compared with the Q-learning, which uses backpropagation algorithm for neural network learning. The comparison was based on achieved fitness and time complexity. Results show, that the best method is HyperNEAT, followed by ES-HyperNEAT and NEAT. Method Q-learning method was least successful in both the achieved fitness and time complexity
ALGORITHMS FOR INTELLIGENT WEB PAGE RANKING
Get PDFCilj diplomske naloge je razviti spletni iskalnik na podlagi lastnega algoritma za inteligentno rangiranje spletnih strani. Iskalnik je implementiran v programskem jeziku PHP, grafično pa se zgleduje po Googlovem iskalniku. Algoritem črpa podatke iz lastne baze mysql, ki je locirana na lokalnem strežniku. V nalogi predstavimo zgodovino iskalnikov, njihovo financiranje in delovanje. S pomočjo programov za analizo rezultatov iskalnikov poskušamo razbrati glavne kriterije, ki vplivajo na razvrščanje spletnih strani v sodobnih iskalnikih. Na kratko še predstavimo optimizacijo spletnih strani in pomen dobrega rangiranja spletne strani na svetovnem spletu.The purpose of this work is to develop a search engine that operates on the basis of its own algorithm for intelligent web page ranking. Search engine is implemented in a programming language PHP, while the graphics is inspired by the Google search engine. The algorithm draws data from its own mysql database, which is located on the local server. In the work we present the history of search engines, their funding and operation. With the help of programs for analyzing the results of search engines we try to figure out the main criteria that affects the classification of web pages in modern search engines. We also briefly present the optimization of websites and the importance of good website ranking on the Internet
DEVELOPMENT OF A STRATEGY GAME WITH THE UNITY GRAPHICS ENGINE
Get PDFV diplomskem delu smo preučili osnove uporabe prosto dostopnega razvojnega orodja in grafičnega pogona Unity. Pridobljeno znanje smo nato uporabili za načrtovanje in razvoj preproste 3D strateške igre v programu Unity. Delo je vključevalo izdelavo 3D modelov s programom Blender, iskanje in prilagajanje brezplačnih zvočnih datotek za glasbeno ozadje igre ter zvočne učinke. Obenem je šlo za programiranje generatorja sveta ter implementacijo funkcionalnosti sistema ekonomije in bojevanja v igri, pripravo uporabniškega vmesnika igre in neprestano sprotno testiranje vseh podrobnosti. Rezultat dela je samostojna računalniška igra, ki jo je možno igrati na operacijskih sistemih Linux, Windows in MacOS.In this thesis, we studied the basics of using the free development tool and graphics engine Unity. The knowledge we aquired was used for the planning and development of a simple 3D strategy game with the Unity engine. The work included making 3D models with Blender, finding and adjusting free sound files for the music background and sound effects, programming a world generator and implementing the functionality of an economy and fighting system, creating a user interface and continuous testing of all the details. The result is a standalone computer game, that can be played on the Linux, Windows and MacOS operating systems
- …
