Engage & Persist

Collaborate

Coding

Image 1

Image 2

Image 3

Physical fabrication

Creature Count is a bird-identifying computer vision project aimed at enhancing birdwatching and aiding scientific research on bird populations. Focusing on Boston's avian diversity, it employs a continuously operating ESP32 camera, encased in durable outdoor housing, to capture detailed data on bird appearances and behaviors. This data is vital for research into bird population trends, migration patterns, and environmental effects. Designed for use in residential backyards, the project serves a dual purpose: enriching birdwatching experiences and providing valuable insights for ecological studies, while being accessible to those unable to venture outdoors.

In this studio, my skills in various areas, particularly software development and team collaboration, saw marked improvement. The project fostered an ideal environment for refining my delegation, communication, and leadership skills. Leveraging my background in computer vision, I provided valuable assistance to team members, enhancing our group's efficiency and knowledge base.

The project's core involved developing a refined YOLO model, programming for camera functionality, and creating a website and database for data visualization. This experience deepened my understanding of training computer vision models using the ultralytics library, managing databases, and preparing datasets. The tech stack for this project included YOLOv8 for computer vision, and a combination of SvelteKit, shadcn-ui svelte, tailwindcss, prisma, lucia auth, planetscale, and cloudflare pages for the web development aspect.

Engage & persist

Collaborate

Coding

Project Paragraph

Growth Paragraph

Physical Fabrication

This studio, I collaborated with fellow NuVu Students, Milin, and Sadie, to create a machine for processing plastic film in recycling facilities. Most recycling facilities are not able to automatically process plastic bags, and resort to using humans to filter this waste from the conveyor belt. Additionally, the plastic film is often missed, and, in turn adulterates the machinery. We began tackling this problem by focusing on the computer vision necessary for detecting plastic bags on a conveyor belt and decided on using the deep learning system, YOLOv8, for training our model. Creating a good computer vision algorithm, even with a system as good as YOLOv8, requires a large dataset of well labeled images, and a lot of compute. We used Roboflow to collect, label and organize a final data set including 9,100 labeled images(post augmentation) and Google Colab Pro which trained the model with 140 epochs(iterations). Originally, we intended for this project to assist the workers sorting plastic film, through a projector which would guide workers to where the plastic was being detected. We felt we could do better, so instead I designed(fusion 360) and we fabricated an arm to remove plastic bags on an also custom make conveyor belt. This arm is equipped with three degrees of motion(if counting the claw), featuring a robust and uniquely designed claw for grabbing plastic film, without getting damaged by other oncoming and not as forgiving materials. This arm is controlled by an Arduino controlled by a python program running on my laptop. This laptop runs our computer vision model using frames from a webcam mounted above the conveyor belt.

This studio, I focused on maintained a mindset of collaboration, through a lot of open communication, and role organization. I would not have been able to do a fraction of what we accomplished without the help and direction from my teammates. This was cultivated by a maintained culture of curiosity, and dedication to the project. Persisting through times of uncertainty was also something I strived for, and during the last 3 weeks. I implemented a simple principle: "The more confused we are, the harder I will work." Because of my supportive teammates and dedication I learned an immense amount of technical skills in programing, and physical fabrication. Winter brake interrupted this studio, but because I was so invested in the project, I spent a fair amount of my break working on it. Prior to the break, our computer vision model was not fully optimized, and the code for the controlling the arm was not written, so I saw an opportunity to retrain the model, and develop this code, when I probably should have been taking a break. Throughout this studio, I re-enforced my recognition of the value of clearly defining the problem and its design requirements, prior to and as a check to ideating solutions.

Collaborate

Engage & Persist

Statistics & data analysis

Project Paragraph

Growth Paragraph

Image 1

Image 2

Image 3

Research

Creature Count is a computer vision bird detector and identifier, and a website for personalized data managing. Esp32 cameras record birds at feeders, and the video goes through the 300 epoch custom YOLOv8 computer vision model, which detects what species the bird is. That data is securely stored in our PlanetScale database and then uploaded onto the Creature Count github and our website, which includes various data visualizations. The project aims to contribute to research and citizen science, as well as being useful to busy birdwatchers who can't sit watching their bird feeder all day but still don't want to miss any birds. It therefore increases the accessibility of birdwatching, promoting an appreciation of nature to a greater population.

Over the course of this studio, I learned more about code and the possibilities of my computer. I installed a lot of things, and explored the possibilities of Terminal, Github, Homebrew, VSCode, Warp, and Xcode, all for the first time. I dipped my toes into Python just a bit and making a variable swapper, tried Orange Data Mining, Google Teachable Machine, and OpeonVC. I also used Observable to set up a species population visualization. A

Engage and Presist

Collaborate

Coding

Project Paragraph

Growth Paragraph

Image 1

Image 2

Image 3

Image 4

Physical Fabrication

Our goal was to create a system that can sort unwanted material out of recycling. Currently workers in recycling facilities are at risk of getting into unsafe environments because of trash bags getting stuck in the machines. To accomplish this goal we designed the EcoSorter, an AI controlled robotic arm that can detect plastic bags and remove them from a conveyer belt. We wanted to build our system out of materials that were cheap and easy to create so this system could be easily be built in many recycling faculties. We created a computer vision algorithm using YOLOv8 (You only look once) and we trained it using a large dataset of labeled images of trash bags. We collected and labeled these images using RoboFlow. Once we trained the model we created a python script to run the model with our web camera. This enabled us to determine the position of each trash bag on the screen, which could then be sent to a system for removal. As the model takes time to detect trash bags, we also estimate their positions while the model is running. Our original plan involved us using a laser pointer which could be used to assist workers but, we eventually decided to build an arm that can push plastic bags of a conveyer while letting other trash go through.

In this Studio, my goal was to comprehend, create, and apply AI algorithms to tackle real-life problems. One of my specific goals was to learn how to create computer vision models to use for AUV which I am working on at beaver. I learned how to create and find datasets and how to use them to train image detection models using Teachable machine, RoboFlow, Yolov8. I also improved my coding ability by debugging problems on the Raspberry pi and creating python scripts with OpenCV. I also explored creating the laser pointer which we scrapped but, I learned how to extract data from the model to send to the Arduino. Overall I went from being not really understanding AI to feeling comfortable creating and training my own algorithms to accomplish a goal. I am excited to keep on using these skills that I learned in future projects such as creating a computer vision system for RoboSub that I will work on once go back to Beaver.

Engage & Persist

Collaborate

Giving & Receiving Feedback

Project Paragraph

Growth Paragraph

Image 4

Coding

Our project was EcoSorter in which we created an AI algorithm to detect plastic bags on recycling sorting facility conveyor belts. A robotic arm at the end grabs any plastic bags that were not sorted out by hand from the workers previously. In recycling sorting facilities, the recycling goes through a preliminary search in which workers manually sort trash out by hand. Then, the recycling goes through a complex system where it is sorted into different materials. If trash is not taken out, specifically plastic bags, rope, and clothing, it can easily get stuck in the machinery and workers have to pull it out which can be dangerous. We made a system with a camera above the conveyor belt that puts a video feed through the AI. The AI then recognizes any plastic bags and gets the coordinates to send a command to the robotic arm about when it will reach the end. The arm grabs the bag and lifts it into the air where it gets sucked up by vacuums that are already in recycling facilities. This would aid workers and make their jobs easier but would not have a risk of taking any jobs away.

Throughout this studio and while working on the project, a consistent goal of mine was to learn applications of Artificial Intelligence and gain an understanding of how it works and processes data. I accomplished these goals because, in the first part of the studio, I learned a lot about programs like Teachable Machine and Orange. When I worked on my project I applied that and I saw the process of training and AI and how that works. I gathered a lot of data from both taking videos and photoshopping images with different backgrounds which I then annotated for it to be trained. I learned how to do some coding in Python which I had never done before. I created an else/if code that produced an output based on input and one based on data from spreadsheets. When working on the project, I grew in my collaboration skills because we had to troubleshoot a lot of things and work together to finish our project in a limited time. I helped our group with time management and making sure we were on track to finish the project. I contributed to the building of the conveyor belt which I worked on for a while and it was one of the elements of the project that functioned well. At the end of the project, I summed up the work we did and created our final presentation which was detailed and taught the audience what we did.

Persist

Collaborate

3D modeling

Project Paragraph

Growth Paragraph

Image 1

Image 2

Image 3

Image 4

Physical fabrication

Creature count is a bird identifying AI which we will use to get more people interested in bird watching. The reason that we ended up choosing birds was because in Boston there are many different species of birds that migrate in and out so having a camera watching at all time will help make sure that you don't miss any birds. In this project we used an ESP32 camera inside of a case so that it could be outside for long amounts of time to help people enjoy the feeling of being in nature even when they can't go outside themself. It does this by using a yoloV8 bird identifying program that will upload identified images to a website The goal of this project is to be a demo of a camera that could be implemented in peoples backyards and could be put next to a bird feeder to watch it at all times. This can also get people more excited to be outside by showing them what they could do outside, inside to possible help people to being more comfortable outside.

During the Creature count studio I feel as though I have grown in many different ways but some of the biggest ones were me getting better and both 3d modeling and physical fabrication while also learning how to collaborate better. On the side of 3d modeling I feel that during Creature count I did very well at learning how to 3d model better even though I already knew how to do the basics by learning how to make sure objects are to scale. I have also practiced a lot of physical fabrication by making different prototypes of my project before I got to my final idea with the 3d-printed items assembled. My main focus when prototyping was making sure that the prototypes were scaled correctly, and I also spent a lot of time working on making sure that after everything fit there wouldn't be to much empty space. We also had to think about the limitations of our data set and making sure that we tested it on birds that were in the data set